erlang os:cmd() command with UTF8 binary - unicode

I'm trying to get an Erlang function to execute a bash command containing unicode characters. For example, I want to execute the equivalent of:
touch /home/jani/ჟანიweł
I put that command in variable D, for example:
io:fwrite("~ts", [list_to_binary(D)]).
touch /home/jani/ჟანიwełok
but after I execute:
os:cmd(D)
I get file called á??á??á??á??weÅ?. How can I fix it?
os:cmd(binary_to_list(unicode:characters_to_binary("touch /home/jani/编程"))).
Executing this command creates a file named ��, while executing the equivalent touch command directly in a terminal creates the file with the correct name.

Its because Erlang reads your source files like latin1 by default, but on newer versions of erlang you can set your files to use unicode.
%% coding: utf-8
-module(test).
-compile(export_all).
test() ->
COMMAND = "touch ჟანიweł",
os:cmd(COMMAND).
and then compiling and executing the module works fine
rorra-air:~ > erl
Erlang/OTP 17 [erts-6.4] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]
Eshell V6.4 (abort with ^G)
1> c(test).
{ok,test}
2> test:test().
[]
and it created the file on my filesystem
rorra-air:~ > ls -lta
total 144
-rw-r--r-- 1 rorra staff 0 Jun 9 15:18 ჟანიweł

Related

bash script calling rdiff-backup never ends

I want to run rdiff-backup and then switch of the raspberrypi it was running on.
I use the following script:
#!/bin/sh
date > /home/mik/rdiff-backup.log
echo "rsync start" >> /home/mik/rdiff-backup.log
rdiff-backup -v5 --print-statistics offlinebackup#server::/srv/backup /srv/datenserverBackup/backup >> /home/mik/rdiff-backup.log 2>&1
sync
date >> /home/mik/rdiff-backup.log
echo "rdiff-backup end" >> /home/mik/rdiff-backup.log
df -h >> /home/mik/rdiff-backup.log
sync
halt
The log file looks good (for the rdiff-backup part):
Sat 12 Aug 08:20:59 UTC 2017
rsync start
Unable to import win32security module. Windows ACLs
not supported by filesystem at /srv/backup
escape_dos_devices not required by filesystem at /srv/backup
Warning: name offlinebackup not found on system, dropping ACL entry.
Further ACL entries dropped with this name will not trigger further warnings
Using rdiff-backup version 1.2.8
Executing ssh -C offlinebackup#server rdiff-backup --server
-----------------------------------------------------------------
Detected abilities for source (read only) file system:
Access control lists On
Extended attributes On
Windows access control lists Off
Case sensitivity On
Escape DOS devices Off
Escape trailing spaces Off
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Unable to import win32security module. Windows ACLs
not supported by filesystem at /srv/datenserverBackup/backup/rdiff-backup-data/rdiff-backup.tmp.0
escape_dos_devices not required by filesystem at /srv/datenserverBackup/backup/rdiff-backup-data/rdiff-backup.tmp.0
-----------------------------------------------------------------
Detected abilities for destination (read/write) file system:
Ownership changing On
Hard linking On
fsync() directories On
Directory inc permissions On
High-bit permissions On
Symlink permissions Off
Extended filenames On
Windows reserved filenames Off
Access control lists On
Extended attributes On
Windows access control lists Off
Case sensitivity On
Escape DOS devices Off
Escape trailing spaces Off
Mac OS X style resource forks Off
Mac OS X Finder information Off
-----------------------------------------------------------------
Backup: must_escape_dos_devices = 0
Starting increment operation /srv/backup to /srv/datenserverBackup/backup
Processing changed file .
Incrementing mirror file /srv/datenserverBackup/backup
Processing changed file abc
Incrementing mirror file /srv/datenserverBackup/backup/abc
Processing changed file abc/def
Incrementing mirror file /srv/datenserverBackup/backup/abc/def
Processing changed file abc/def/testfile.dxf
Incrementing mirror file /srv/datenserverBackup/backup/abc/def/testfile.dxf
--------------[ Session statistics ]--------------
StartTime 1502526061.00 (Sat Aug 12 08:21:01 2017)
EndTime 1502527913.72 (Sat Aug 12 08:51:53 2017)
ElapsedTime 1852.72 (30 minutes 52.72 seconds)
SourceFiles 151099
SourceFileSize 386321558216 (360 GB)
MirrorFiles 151097
MirrorFileSize 386321447731 (360 GB)
NewFiles 2
NewFileSize 110485 (108 KB)
DeletedFiles 0
DeletedFileSize 0 (0 bytes)
ChangedFiles 1
ChangedSourceSize 0 (0 bytes)
ChangedMirrorSize 0 (0 bytes)
IncrementFiles 4
IncrementFileSize 0 (0 bytes)
TotalDestinationSizeChange 110485 (108 KB)
Errors 0
--------------------------------------------------
The backup is working, but then the script ends right there.
rdiff-backup.log contains the full report of rdiff-backup. But neither the line "rdiff-backup end", nor the output of "df -h".
How can I make it ran to the end?
Thanks for your answers
I finally found a workaround, that solves my problem.
My sciprt which is called after booting from /etc/init.d is calling the other script which does the actual work (i.e. backup my data, and write the log file) as a background task.
/etc/init.d/CallAfterBoot.sh
#!/bin/sh
sleep 30
/home/me/DoBackup.sh & # '&' starts the script in background
/home/me/DoBackup.sh is the script I posted above which is now runing correctly.
Same script running as the same user now behaves differently. There's got to be some bug somewhere, however, it works for me now.

Wierd behavior of db2ls when run from directory containing spaces

I am facing a strange issue with getting the db2 version using db2ls.
Below are 2 instances of db2ls execution
[root#dummy 6]# cd /tmp
[root#dummy tmp]# db2ls
Install Path Level Fix Pack Special Install Number Install Date Installer UID
---------------------------------------------------------------------------- -----------------------------------------
/opt/ibm/db2/V10.5 10.5.0.3 3 Tue Mar 10 01:38:53 2015 PDT 0
[root#dummy tmp]# mkdir test\ dir
[root#dummy tmp]# cd test\ dir/
[root#dummy test dir]# db2ls
/usr/local/bin/db2ls: line 43: cd: /tmp/test: No such file or directory
Install Path Level Fix Pack Special Install Number Install Date Installer UID
---------------------------------------------------------------------------------------------------------------------
/opt/ibm/db2/V10.5 10.5.0.3 3 Tue Mar 10 01:38:53 2015 PDT 0
It looks like db2ls is having issues when executed from a directory having spaces. Is this a known issue? I could not find any documentation for this. I am trying to circumvent this problem by using db2ls 2>/dev/null.
If there is a more efficient way please let me know.
http://www-01.ibm.com/support/knowledgecenter/SSEPGG_10.5.0/com.ibm.db2.luw.qb.server.doc/doc/t0023683.html
for DB2 installation paths it says
DB2 installation paths have the following rules:
Can include lowercase letters (a-z), uppercase letters (A-Z), and the underscore character ( _ )
Cannot exceed 128 characters
Cannot contain spaces
Cannot contain non-English characters
Reading statements like this are always a warning sign not to use spaces in paths in general. BTW, shell scripts don't play well with spaces in paths which is for me a good reason to avoid spaces in general.

cdb/windbg output too large

We are using cdb (command line version of winDBG) to resolve a cab.
For getting the output in a file we are using the -logo output to specify the output file.
For a certain cab we are getting "CvRegToMachine(x86) conversion failure for 0x7536" more than a million times.
Basically we get a huge resolved code log, nearly 1GB, and all of it filled with the above string on each line.
We are using the following cdb command
cdb -z "abc.cab" -y "SymbolsPath" -G -logo "outputfile" -lines -c ".kframes 100;!analyze -v;!load msec.dll;!exploitable -v;vertarget;lmv;q"
Does anyone have any clue about what could be wrong here?
you cannot use -c and -G at the same time -c needs the first break to read the initial
command and act upon it if you need to run the code use g; at the end of -c commands
also many times -c commands need to be provided first and the debugee needs
to be at the end of commandline
cdb -c "<some cmd;someother cmd;g>" -z foo.cab
.load not !load should be used to load third party extensions
the string emitted "CvReg.........." seems to be related to Either SYMFLAG_NULL or SYMFLAG_REGISTER in the Flags member of SYMBOLINFO Struct .
a sample trial didnt cross that code path in my machine so either the corrupt dmp or more information regarding the dmp file may be needed to find the reason for the spew
creating dump
C:\>dir /b *.cab
File Not Found
C:\>cdb -c ".dump /ma /b foo.cab;q" calc | grep -i -E "dmp|dump|wr"
0:000> cdb: Reading initial command '.dump /ma /b foo.cab;q'
Creating C:\DOCUME~1\Admin\LOCALS~1\Temp\foo.cab.dmp - mini user dump
Dump successfully written
Adding C:\DOCUME~1\Admin\LOCALS~1\Temp\foo.cab.dmp - added
Wrote foo.cab
C:\>dir /b *.cab
foo.cab
**loading dump as debugees(cdb) debuggee (foo.cab) and looking around **
C:\>cdb cdb -z foo.cab
0:000> s -u dbgeng l?39b000 "CvReg"
020341f8 0043 0076 0052 0065 0067 0054 006f 004d C.v.R.e.g.T.o.M.
0:000> # *(*20341f8 dbgeng l?39b000
dbgeng!MachineInfo::CvRegToMachine+0xfe:
021bf8ae 68f8410302 push offset dbgeng!`string' (020341f8)
0:000> # call*dbgeng!MachineInfo::CvRegToMachine dbgeng l?39b000
dbgeng!ImageInfo::CvRegToMachine+0x22:
021b62f2 e8b9940000 call dbgeng!MachineInfo::CvRegToMachine (021bf7b0)
0:000> # call*CvRegToMachine dbgeng l?39b000
dbgeng!ImageInfo::CvRegToMachine+0x22:
021b62f2 e8b9940000 call dbgeng!MachineInfo::CvRegToMachine (021bf7b0)
dbgeng!TypeInfoValueFromSymInfo+0x4b:
022541ab e82021f6ff call dbgeng!ImageInfo::CvRegToMachine (021b62d0)
dbgeng!TypedData::SetToSymbol+0x25f:
02285edf e8ec03f3ff call dbgeng!ImageInfo::CvRegToMachine (021b62d0)
dbgeng!TypedData::SetToSymbol+0x2da:
02285f5a e87103f3ff call dbgeng!ImageInfo::CvRegToMachine (021b62d0)
0:000> ln 0x21bf8ae
(021bf7b0) dbgeng!MachineInfo::CvRegToMachine+0xfe | (021bf8d0) dbgeng!Ma
chineInfo::GetContextState
doing and uf on this function yields the check where SymbolInfo->Flags is
checked and decided my sample dump above doesnt enter the path

OmniORB compilation error Windows 7 64 bit

Has anyone encountered the error below when compiling omniORB_4.1.6 64-bit for windows?
'RegQueryValueEx failed - error 109'
I followed the procedure in the readme.win32 and I get linking errors in the omniDyamic, codesets etc.. So someone suggested to rebuild the omniorb_root/src/tools/win32 and copy it in bin/x86_win32/. That's what I did and when I recompile the whole omniORB, the error is as below:
../../../../bin/x86_win32/omkdepend -D__cplusplus -D_MSC_VER -DIDLMODULE_VERSION
="0x2630" -DMSDOS -DOMNIIDL_EXECUTABLE -Ic:/python27/include -Ic:/python27/PC -I
c:/python27/include/python2.7 -DPYTHON_INCLUDE=<Python.h> -I. -I. -I../../../../
include -D__WIN32__ -D_WIN32_WINNT=0x0501 -D__x86__ -D__NT__ -D__OSVERSION__=4 -
D_CRT_SECURE_NO_DEPRECATE=1 idlc.cc idlpython.cc idlfixed.cc idlconfig.cc idldum
p.cc idlvalidate.cc idlast.cc idlexpr.cc idlscope.cc idlrepoId.cc idltype.cc idl
util.cc idlerr.cc lex.yy.cc y.tab.cc
RegQueryValueEx failed - error 109
-----------------------------------------------------------------------------------------------
make[4]: Entering directory `/cygdrive/c/Software/COTS/omniORB/omniORB_4.1.6/src
/tool/omniidl/cxx/cccp'
../../../../../bin/x86_win32/clwrapper -gnuwin32 -c -O2 -MD -GS -GR -Zi -nologo
-DHAVE_CONFIG_H -I. -I. -I. -I../../../../../include -D__WIN32__ -D_WIN32_WINNT=
0x0501 -D__x86__ -D__NT__ -D__OSVERSION__=4 -D_CRT_SECURE_NO_DEPRECATE=1 -Focexp
.o cexp.c
RegQueryValueEx failed - error 109
I'm going to answer my own question because it seems nobody has encountered this problem, and the mailing list is so quiet.
Someone suggested to me to recompile the src\tools\win32. So that's what I did and I copied the .exe files generated to bin\x86_win32.
I then compiled all the omniORB and get the RegQueryValueEx error.
The reason for this is when you check the src\tools\win32\bccwrapper.c in the void GetMounts(void) function,
it looks for this path in the registry:
Software\Cygnus Solutions\CYGWIN.DLL setup\b15.0\mounts\%02X.
When I checked that using regedit, I noticed that in the mounts->00, 01, 02, 03 etc.. keys, there are no 'unix' and 'native' string values inside those keys.
So I decided to delete all the keys and retained just the 00 and added a 'unix' and 'native' string value.
After which, I recompiled the src\tools\win32 and copied over the created .exe files to bin\x86_win32 and finally when I recompiled all the omniOrb, it started compiling (need to copy the ssl libs too) and finished successfully.
I really don't even know how the following got into my registry:
Software\Cygnus Solutions\CYGWIN.DLL setup\b15.0\mounts\%02X.
Best regards,
Mark
I spent quite some time trying to compile OmniORB on windows 10 with visual studio 2017.
Assuming Cygwin64 was installed in directory
c:\software\cygwin64
, the compilation of OmniORB is quite straightforward:
open a command terminal (cmd)
in that terminal, setup the Visual environment:
"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvarsall.bat" x64
then, append the PATH (yes append and not prepend):
set PATH=%PATH%;c:\software\cygwin64\bin
then, in file config\config.mk, uncomment this line
platform = x86_win32_vs_15
in file platforms\x86_win32_vs_15, set PYTHON to target the python executable, in my case Python 3.6.5
PYTHON = /cygdrive/c/software/Python/python
finally start the compilation with make:
make export
Hope this helps.

Schrödinger's file

I am puzzled by the following sequence of commands.
sh-4.2$ pwd
/home/willard
sh-4.2$ ls -l f
-rwxr-xr-x 1 willard users 59116 Jan 23 14:54 f
sh-4.2$ file f
f: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.15, BuildID[sha1]=0xea0e08ff2b5a062698d45b78177acdd6bf140d1f, stripped
sh-4.2$ ./f
sh: ./f: No such file or directory
sh-4.2$ strace ./f
execve("./f", ["./f"], [/* 32 vars */]) = -1 ENOENT (No such file or directory)
write(2, "strace: exec: No such file or di"..., 40strace: exec: No such file or directory
) = 40
exit_group(1) = ?
+++ exited with 1 +++
sh-4.2$ ls -l f
-rwxr-xr-x 1 willard users 59116 Jan 23 14:54 f
sh-4.2$ uname -a
Linux xdat10 3.6.2-1-ARCH #1 SMP PREEMPT Fri Oct 12 23:58:58 CEST 2012 x86_64 GNU/Linux
How is this possible?
I found someone having the same problem (with relative explanation)
Running 32bit binary on a 64bit system
Quoting the most important sentences:
This situation often arises when you try to run a binary for the right
system (or family of systems) and superarchitecture but the wrong
subarchitecture. Here you have ELF binaries on a system that expects
ELF binaries, so the kernel loads them just fine. They are i386
binaries running on an x86_64 processor, so the instructions make
sense and get the program to the point where it can look for its
loader. But the program is a 32-bit program (as the file output
indicates), looking for the 32-bit loader /lib/ld-linux.so.2, and
you've presumably only installed the 64-bit loader
/lib64/ld-linux-x86-64.so.2 in the chroot.
You need to install the 32-bit runtime system in the chroot: the
loader, and all the libraries the programs need. On Debian amd64, the
32-bit loader is in the libc6-i386 package. You can install a bigger
set of 32-bit libraries by installing ia32-libs.
I bet there's a better way to verify this but i'd try to exec
ldd ./f
and search in the output which loader is needed to exec'it
man 2 execve:
ENOENT The file filename or a script or ELF interpreter does not
exist, or a shared library needed for file or interpreter can‐
not be found.
You could run ldd against this binary to look for libraries that could not be mapped and install them from multilib.