I have two files that give the same hash, and even the same hexdump. File A and File B start on Linux Box 1 and Linux Box 2, respectively. I then copy both files to a Windows share, and read them from a Windows machine. The files still seem to be byte-by-byte identical with the Windows utility Fc (with /b option -- binary mode). However, when I open the two different files, they appear to have different encoding (newlines/line-wrap). Why wasn't this uncovered by the hashes/hexdump/Fc?
What am I overlooking here?
Don't use wordpad for that. Actually, don't use wordpad at all. Note that Microsoft often does not keep to standards, and in many times (e.g. the browser) simply takes and informed guess at file or stream content, using the header as some kind of magic. Sometime it guesses wrong, some times it doesn't.
You could calculate the hash on the Windows machine as well, there are plenty of lightweight utilities that calculate secure hashes within windows Explorer. You could also install command line utilities such as OpenSSL on Windows (or take it a step further and install Cygwin, which I always have running on my Windows machine).
Windows has never had a real strategy regarding line endings, except keeping to it's own double-character standard. In later versions of Windows you may use Notepad which does (finally) understand Unix newlines if you must (because maybe it screws up UTF-16 this time around).
Related
The question I'm asking is a bit unclear, I think it's a bit hard to explain it with only one line. Here's my situation: I have a Bitbucket repertory which is cloned in both an Linux environment and Windows environment. The problem I have is:
1- I need to read and write from files and the paths to the different locations must be changed every time I commit and push. Hence, if I was working on Windows and made a push, when I go back to Linux and need to pull and change the paths I used.
2- I'm running selenium with Python. In order to make it work on my Linux serverless machine, I need to create a virtual display with pyvirtualdisplay library. Hence, some code that needs to be executed on my Linux machine must not be executed on my Windows machine.
So the problem I got is if I work on my Windows machine, I need to comment out the lines that creates the virtual display.
These two problems takes me a lot of time, because everytime I pull in a different machine, I can't directly work on the code, but have to change the code first.
Pretty simple. What you are looking for is to import the 'os' module.
import os
Then you can get the "name" of the current running Operating System by checking: os.name.
You can then base if or case statements, or however you want to manage your code, on what OS is reported.
To be clear, I would wrap the code I want to run only in a certain environment in an if statement that determines which OS is running and runs the appropriate code block based on the result.
if [[ os.name == "nt" ]]; then
;do Windows stuff
else
;do linux stuff
fi
Additional Note! That was psuedocode based on general Bash scripting.
This is Python psuedocode:
if os.name == "nt":
; Do Windows Stuff
else:
; Do Linux Stuff
Your first issue can be solved by handling the paths in a configuration file instead of hard-coding them in the program itself (and possibly using the pathlib or os.path modules for the actual path manipulations), then telling git to ignore the config file. You can then create appropriate configurations on each system, and git won't bother them at all.
Your second issue can be solved by using any of the various methods Python gives you to figure out what OS you're running on, and then using simple conditional statements to match on them. In theory, you could do the same for the paths, but it really is better in the long run to get in the habit of properly separating runtime configuration like that from program logic. Options for this include:
os.name: Contains posix, nt, or java, which identifies what type of OS you're on. java for Jython, nt for Windows,posix` for pretty much everything else. Useful when you just care about certain low-level OS semantics.
sys.platform: Contains a generic name for the underlying OS ABI. win32 for Windows, darwin for macOS, linux for Linux, and the name of the OS for other UNIX variants. This lets you check for particular underlying platforms, and is generally what you should use for conditional code that only runs on one platform. Make sure to always check this with a construct like sys.platform.startswith('X'), as some platforms and Python implementations include version info after the OS name.
platform.system(): Similar to sys.platform, except that the returned string is more user-friendly (Windows for Windows, Linux for Linux, etc), and it returns an empty string if it can't figure out what OS you're on. Useful for displaying the OS to the user, but not for doing conditionals (because it's free-form and may not always return useful information).
I calculated MD5 and SHA256 hash values of notepad.exe and mspaint.exe through online hash generators md5FileCalculator Onlinemd5.
What i noticed is that if i calculate when both exe's are present in their actual postion in system32 the value coming is different than when placed somewhere out of system32 folder.
What is the reason behind that ? Which is the correct hash value ?
I am using Software Restriction policy to block the applications, I created a hash rule for notepad.exe(present in SYSTE32 folder) file and blocked it. When I check the hash value in registry it is different from the hash value of notepad.exe (from SYSTEM32 folder) calculated through other methods like online md5 calculators or through Windows API. But when I copy the notepad.exe file into some other folder say on desktop and calculate the hash value, it is coming same as it is in registry for which I created the rule.So the correct value is I think the one which I get when file is out of system32 folder. But I am not getting why it is happening ? Does it have something to do with permissions ?
It's because of 32-bit applications running on 64-bit Windows, and how Windows handles the System32 folder for those programs.
This was also driving me nuts for a while because I couldn't for the life of me figure out why certain files in System32 (namely .dlls and .exes) were returning different hashes depending on what I checked them with.
Using HxD and Firefox to upload a file to check its hash, I got different results compared to using QTTabBar's hash checker, which runs inside explorer.exe.
But if I copied one of these files to another location, I would then get identical results across all programs.
Meanwhile, HxD showed different file lengths for the copied file vs the one in System32, and while both showed similar byte distribution, there were also significant differences.
But then I thought to try the same thing on another folder, and finally cracked it, with a little help from Wikipedia:
The operating system uses the %SystemRoot%\System32 directory for its
64-bit library and executable files. This is done for backward
compatibility reasons, as many legacy applications are hardcoded to
use that path. When executing 32-bit applications, WoW64 transparently
redirects 32-bit DLLs to %SystemRoot%\SysWOW64, which contains 32-bit
libraries and executables.
32-bit applications are generally not aware
that they are running on a 64-bit operating system. 32-bit
applications can access %SystemRoot%\System32 through the pseudo
directory %SystemRoot%\Sysnative.
Because HxD and Firefox (and most other browsers) are all 32-bit applications, when you load a file into them, Windows is actually transparently redirecting them to the file of the same name in the SysWOW64 folder (presumably if you ran a 64-bit browser, you would not encounter this problem).
Similarly, when you copy a file out of System32 to another location, explorer.exe, being a 64-bit process, copies the original System32 file, and not the (confusingly named) SysWOW64 equivalent.
So as the wiki states, if you enter %SystemRoot%\Sysnative
into the path of the open file dialogue in your 32-bit application, it should load the file from the real System32 folder, and give you the correct result.
And if you check the files in the SysWOW64 directory, all files should return the same respective hashes regardless of what you open them with.
Further reading:
SysWOW64
Sysnative
Are you sure you're checking the exact same file yet on diferent paths? I think you're checking two diferent notepad.exe. Check the size of the file... it must be exacly the same on bytes.
I've just checked my notepad.exe on two distinct paths C:\Windows\System32 and C:\Windows and they are diferent.
I installed (extracted) msysgit portable (PortableGit-1.9.5-preview20150319.7z)
The compressed archive is 23 MB, but once extracted the contents take up 262 MB. This is mostly due to the git command binaries (under 'libexec\git-core'). Almost all of the binaries are identical, they just have different names.
Why did the developers build the project like this? I suppose they need an executable for each command to support the CLI on windows cmd.exe.
But isn't there a way to avoid having ~100 identical binaries, each 1.5 MB in size (ex: using batch files)?
Why did the developers build the project like this? I suppose they
need an executable for each command to support the CLI on windows
cmd.exe.
Under unixoid OSes, you can have symbolic links to another file that behave exactly like the original file; if you do that for your executable, your executable can look into argv[0] to find out how it was called. That's a very common trick for many programs.
Under Windows, and especially without installers, it's (to my knowledge) impossible to get the same behaviour out of your file system -- there's just no symbolic link equivalent. Especially when you consider that programs that are meant to run from USB drives have to cope with ancient filesystems such as FAT32!
Thus, the executables simply were copied over. Same functionality, more storage. However, on a modern machine that you'd run windows on, you really don't care about 200MB give or take for such a versatile tool such as git.
In conclusion: the developers had no choice here; since windows (though having some posix abstraction layer) has no proper filesystem support for symbolic links, that was the only good way to port this unix-originating program. You either shouldn't care or use an OS that behaves better in that respect. I'd recommend the latter, but OS choices often aren't made freely...
What options are there for saving and retrieving documents to and from the cloud, from within Emacs?
I use Emacs at work, on a Windows machine, and at home, on a Linux box, so ideally I would want a solution that works more or less out of the box for both operating systems.
I touched on g-client, but could not quite get it to work. Obviously, if there are no other, simpler options, I'm just going to have to spend a couple of more hours on it.
Many thanks,
Andreas
Dropbox is pretty universal. I store even my Emacs config files there. Works on Windows, Linux, OS X, and iPhone. Syncs automatically. Stores history. Is free. What else do you want?:-)
Two options that I can think of:
If you have access to a server somewhere that runs ssh, then use ssh with tramp. You can also run a ssh server at your home linux box and access your home files through from work. Tramp works perfectly fine on Windows with ssh from cygwin. It will automatically grab a file (provided that you give emacs something like /ssh:yourusername#yourserverhost:~/yourfile), put it to a temporary file at your computer, then copy it back to the host when you save it.
Use a source control system like SVN or Git. Again you can host the server at your home or you can find online hosts (most are for open source and are thus public, but some are free and private; I use unfuddle.com). You would have to regularly commit/update, but you can easily automate that if you want, and the source control system gives you a nice history of your files and a safety net in case you did something very wrong.
Emacs has excellent integration with source control system. If you find the build-in one not sufficient (it is quite generic and thus does not offer interface to some specific features of a particular source control system), there are plenty of good alternative (psvn for SVN, and magit for Git, for example).
sshfs, if you have good connection speed.
Otherwise there's always tramp-mode for Emacs.
Edit: Just saw you are using Windows.
It's been some years since I used Windows as my desktop, but I used WebDrive back then. It sort-of works, although it always was a bit unstable.
Emacs has great support for remote file systems via Tramp. So the real question is what should you use as a remote FS. There are a bunch of them and as long as they have a way of mounting them or logging in via ssh (for Tramp) you should be ok.
I use JungleDisk - works great for Windows, Linux and Mac. Starts around $2 per month and there's a cap of around $90 per year. You can back up to S3 or to Rackspace.
It integrates at the file system level so you can either read/write directly to it or create links from it to your local file system. I use that to share my .emacs, .bash etc between multiple machines.
Chris
I want to run some system commands (to fix things) before running an executable. I have a reasonably locked down (work) Windows XP system and so can't change what a shortcut points to. For my users' convenience, I must keep the same shortcut. However, I am able to swap out the .exe (renaming) and potentially replace it with another .exe (of the same name) which runs my system commands and then runs the original .exe.
What would be the easiest and quickest language/compiler to do this in? Previously, I've done this sort of thing in C (and tried it today in Python using py2exe without much success). Preferably free solutions.
Visual C# 2008 Express Edition is
free
comes with a compiler
outputs exes
C# is a good choice if you have C
experience
.net currently is the "canonical"
Windows platform