In programming how do you go about reading raw data off of disks. Note: not with a hex-editor, I know how to do that. I basically want to make my own tool.
For example; I want to be able to read the raw data off of a flash drive or some other disk so I can find deleted data. Is it as simple as opening a file and reading a stream? Can someone point me in the right direction?
Obviously I would want the data to appear in hex so that I can scan for file signatures (http://www.garykessler.net/library/file_sigs.html)
C and Python are the ones I am really curious about. Will the standard libraries allow you to open a disk and read data from it directly?
Linux and Windows are two OS's I use.
Thanks
Related
I want to know if i can extract data from .dat file imitating the way I do it inside the software.
Commonly, I load the .dat file in the software through "Load Data" option, then is on my interest obtaining 2 files, generated through "Params to ASCII" and "Data to ASCII" options inside the software. Like you see, I obtain 2 ascii files, which are easily read with a text editor.
The concern is that I do it all manually, and there are lot of .dat files, so I spend lot of ass-hours doing just clicks.
So, I want to know if there is some way to automatize those operations, anyway serve. I am thinking, through my limited knowledge, in scripts that imitate what I do manually (don't know how to do it), or something more complex, which involves reverse-engineering (also don't know how to do it or if it's possible). Or maybe using powershell...
Maybe you guys could help me, surely you have more brillaint minds!
Kind regards!
There are at least four options that I can think of. Sadly, .dat is not a well-defined file format like .pdf, but a general extension used for all kinds of data files. Do you know the name of the software you open the files? That would help to find a solution. Anyway, some general ideas; to recommend any or be more practical requires to know the software.
Use application vendor's API or libraries to read the file. Vendors often provide .Net library for reading the file from disk or via API call. This would be the clean and supported way. For example, to read dBase database files, there's a library at Github.
Read the file as raw binary (as explained in the article linked by Abraham Zinala). I'd rather not try this first, as it requires some reverse engineering and might provide unexpected errors.
Use UI automation. That is, create a script that uses SendKeys to simulate pressing keyboard keys. There are tools such as AutoIT that make this easier. This is kind of last resort, as it is error prone and cumbersome. If the software supports macros or has internal automation capability, try that before 3rd party tools.
The system sending you .dat files offers the data in some other easy to process format. Whilst this is the easiest solution for you, the other party might not agree.
I am new to exploit development, and I am wondering how I could go about fuzzing a local application (e.g. Windows Media Player) using software if I did not have access to the source code. I have basic-intermediate knowledge of buffer overflow vulnerabilities, how they work, how to find them and how to write exploits for them. I just need the 2nd step - finding them in the first place...
Thanks!
Fuzzing is a useful tool but not necessarily easy. I'd suggest starting with listening to some talks online on fuzzing or doing some reading. The big picture is that you want to find a way to generate random/mutated input and then script your application to run with this random input. So for Windows Media Player, you might start by:
Figuring out how to run WMP in an automated/scripted way.
Looking for tools to generate fuzzed file input.
I am trying to use Spray to download a medium-sized file, say a video, from my Scala app.
What I need is to download the file and, for every chuck that I receive (possibly the chuck dimension is something I would set via config), to track the progress of the whole process (a.k.a. progress-bar like).
Additional, I would like to avoid to have the whole file in memory before saving it, instead saving each chuck as I receive it (and log it).
I looked at the documentation, and a bit online.
However, I can't seem to find a clear way of doing this.
The only thing I found was this link which says that this use-case was not supported by spray-client at the time and I should use spray-can directly...no example though, so I am still a bit confused.
Can anybody suggest an approach? Point me in the right direction?
How would you download a file of, say, 30MB?
Thanks
I alongwith some friends of mine was having this idea of creating a very basic media player,Say it only plays mp3s(very limited GUI showing the controls of the player).The Tricky part is we wish to do it without any OS.i.e It will be loaded directly from the boot loader and we will read the file off from a pen drive.Is this a feasible project idea.Its a hobby project so there is no time limit as of now. Can someone guide me as to how to proceed?
If you want to do this from scratch, all I can say is good luck. Yu'd have to write file system drivers, sound drivers, display drivers, keyboard drivers, mouse drivers, USB drivers, etc. it would be a massive amount of work.
Instead, if you still want to write things like the mp3 codec yourself, you can, but do it on top of an existing Os. You could go with DOS if you wanted, and have at least something built in to work off of. If I were doing this, I'd be going with embedded Linux + x11. But it's your choice.
I don't just mean publish, but pretty much everything between when the pure coding is finished and the first version is released. For example, how do games make it so that their save files are hidden/unhackable, how do they include their resources within the game as opposed to having a resource file containing all of the sprites, etc., how do they make it so that there are special file extensions like .rect and .screen_mode, and so on and so forth.
So does anyone know any good books, articles, websites, etc. that explain the process between completing the pure code for a game and the release of it?
I don't think developers make much of an effort to ensure saves are hidden or unhackable. PC games usually just save out to a folder, one file per save, and any obfuscation is likely the result of using a binary file format (which requires some level of effort to reverse-engineer) or plaintext values that aren't very meaningful out of context, but not deliberate attempts to circumvent hacking. There are probably a ton of PC games that have shipped with very easily hackable text or XML save files, but I've never been a save hacker so I don't have any specific examples. On consoles the save files are going to a memory card or the console's hard drive, which makes them inherently inconvenient to access, but beyond that I don't think console developers make much of an effort to encrypt or otherwise obfuscate save data. That energy would more likely be directed towards securing the game against cheating if it's on online game or just making other systems work better.
Special file extensions come from just using your own extensions and/or defining your own file formats. You can use any extension for any file, so there are tons of "special" file formats that are just text files with a different extension, I've done this plenty of times myself. In other cases, if they have defined their own binary file format, that means they also have their own file parsers to process those files at runtime.
I don't know what platforms you have in mind, but for PC and console games, resources are not embedded in the executable. You will generally see a separate executable and then various archives and configuration files. Depending on the game, it may be a single resource pack, or perhaps a handful of packs for related resources like graphics, sound, level data, etc. As a general observation console games are more aggressively archived (to minimize file operations on slow optical media, and perhaps to overcome limitations of the native file systems on more primitive platforms). Some PC games have very loose assets, with even script files hanging out in the open.
If you develop for Windows or XBox 360, Microsoft might offer some help here. Check out their Game Development tools for Visual Studio C++ Express Edition.
If you are looking for books the Game Development Essentials series should answer your questions.
For circumventing saved file modifications, you can implement a simple encryption algorithm and use it to encrypt saved files, and then decrypt them when loading. File extensions are simply a matter of choice.
To use special file extensions in your game, just do the following:
Create some files in a format of your choice that have that extension, and then
write some code that knows how to read that format, and point it at those files.
File extensions are conventions, nothing more; there's nothing magic about them.
ETA: As for embedding resources, there are a few different ways to approach that problem. One common technique is to keep all your resources bundled together in a small number of files - maybe only one (Guild Wars takes that approach).
At the other extreme, you can leave your resources spread across many files in a directory tree, maybe in a custom format that requires special tools to modify, and maybe not. Civilization 4 does things this way, as do all the Turbine games I'm familiar with. This is a matter of taste, and not very important either way.
I think a better solution is two break your images in tiles of some known size and then join them back to back in some random order in a new file. This random order is only known to you and hence only you know how to jumble the tiles to get the original image back.
The approach would be to maintain a single dimensional array and maintains the position of tiles in it. Know use the crop functions of MIDP to extract each tile and render each tile back to the console.
If you need, I can post the code for you.
I would suggest to check the presentation from the developers of World of Goo (great game):
http://2dboy.com/public/eyawtkagibwata.pdf.