Can anyone tell me what is the difference between SSL_CTX_set_cert_verify_callback and SSL_CTX_set_verify?
From OpenSSL docs:
SSL_CTX_set_cert_verify_callback() sets the verification callback function for ctx. SSL objects that are created from ctx inherit the setting valid at the time when SSL_new(3) is called.
and:
SSL_CTX_set_verify() sets the verification flags for ctx to be mode and specifies the verify_callback function to be used. If no callback function shall be specified, the NULL pointer can be used for verify_callback.
So I'm trying to understand which callback to send for each one (from client side).
Thanks experts.
SSL_CTX_set_cert_verify_callback() means you're specifying a function to do the entire validation process (walking the certificate chain validating each cert in turn). [ you probably don't want to be doing this, per the warning below ]
SSL_CTX_set_verify(), on the other hand, specifies a function that's called when the default validator checks each certificate, with preverify_ok set to 0 or 1 to indicate if verification of the certificate in question worked.
From the doc for SSL_CTX_set_cert_verify_callback()
WARNINGS
Do not mix the verification callback
described in this function with the
verify_callback function called during
the verification process. The latter
is set using the SSL_CTX_set_verify(3)
family of functions.
Providing a complete verification
procedure including certificate
purpose settings etc is a complex
task. The built-in procedure is quite
powerful and in most cases it should
be sufficient to modify its behaviour
using the verify_callback function.
SSL_CTX_set_cert_verify_callback() changes the default certificate verification function. You probably should not do this. It's quite involved, you need to check the signature for each cert, verify the chain, possibly check CRL. It's the most complicated part of the SSL.
The SSL_CTX_set_verify() is used to set the mode of SSL. If the mode is SSL_VERIFY_PEER (2-way SSL), you should also set a callback in this function to further verify the client certificate (checking CN against a white-list etc). For other modes, this CB is not used. Since you said you are in client mode, you probably don't need to worry about this call.
Related
I'm using hunchentoot session values to make my server code re-entrant. Problem is that session values are, by definition, retained during the session, i.e., from one call from the same browser to the next, whereas what I really am looking for is what amount to thread-specific re-entrancy, so that all the values disappear between calls -- I want to treat each click as a separate "from scratch" event, even if they are from the same session . Easy enough to have the driver either set to nil, or delete my session values, but I'm wondering if there's a "correct" way to do this? I don't see any thread-based analog to hunchentoot:session-value in the documentation.
Thanks in advance for any guidance you can offer.
If you want a value to be "thread specific" and at the same time to be "from scratch" on every request, that requires that every request must be dispatched in a brand new thread. This is not the case according to the Hunchentoot documentation, which says that two models are supported: a single-threaded taskmaster and a thread-per-connection taskmaster.
If your configuration is multi-threaded, then a thread-specific variable bound in a request-handling can therefore be expected to be per-connection. In a single-threaded Hunchentoot setup, it will effectively be global, tied to the request servicing thread.
A thread-based analog to hunchentoot:session-value probably doesn't exist because it would only introduce behaviors into the web app which surprisingly change if the threading model is reconfigured, or if the request pattern from the browser changes. A browser can make multiple requests using the same connection, or close the connection between requests.
To extend the request objects with custom per-request, I would look into, perhaps, subclassing from the acceptor (how to do this is described in the docs). My custom acceptor would have a custom method of the process-connection generic function which would create extended/subclasses request objects carrying the extra stuff I wanted to put into a request.
Another way would be to have some global weak hash which binds request objects as keys to additional information.
I've been reading a lot about OpenSSL, specifically the TLS and DTLS APIs. Most of it makes sense, it's a pretty intuitive API once you understand it. One thing has really got me scratching my head though...
When/why would I use BIOs?
For example, this wiki page demonstrates setting up a barebones TLS server. There isn't even a mention of BIOs anywhere in the example.
Now this page Uses BIOs exclusively, not ever using the read and write functions of the SSL struct. Granted it's from 2013, but it's not the only one that uses BIOs.
To make it even more confusing this man page suggests that the SSL struct has an "underlying BIO" without ever needing to set it explicitly.
So why would I use BIOs if I can get away with using SSL_read() and SSL_write()? What are the advantages? Why do some examples use BIOs and others don't? What Is the Airspeed Velocity of an Unladen Swallow?
BIO's are always there, but they might be hidden by the simpler interface. Directly using the BIO interface is useful if you want more control - with more effort. If you just want to use TLS on a TCP socket then the simple interface is usually sufficient. If you instead want to use TLS on your own underlying transport layer or if you want have more control on how it interacts with the transport layer then you need BIO.
An example for such a use case is this proposal where TLS is tunneled as JSON inside HTTPS, i.e. the TLS frames are encoded in JSON and which is then transferred using POST requests and responses. This can be achieved by handling the TLS with memory BIO's which are then encoded to and decoded from JSON.
First, your Q is not very clear. SSL is (a typedef for) a C struct type, and you can't use the dot operator on a struct type in C, only an instance. Even assuming you meant 'an instance of SSL', as people sometimes do, in older versions (through 1.0.2) it did not have members read and write, and in 1.1.0 up it is opaque -- you don't even know what its members are.
Second, there are two different levels of BIO usage applicable to the SSL library. The SSL/TLS connection (represented by the SSL object, plus some related things linked to it like the session) always uses two BIOs to respectively send and receive protocol data -- including both protocol data that contains the application data you send with SSL_write and receive with SSL_read, and the SSL/TLS handshake that is handled within the library. Much as Steffen describes, these normally are both set to a socket-BIO that sends to and receives from the appropriate remote host process, but they can instead be set to BIOs that do something else in-between, or even instead. (This normal case is automatically created by SSL_set_{,r,w}fd which it should be noted on Windows actually takes a socket handle -- but not any other file handle; only on Unix are socket descriptors semi-interchangeable with file descriptors.)
Separately, the SSL/TLS connection itself can be 'wrapped' in an ssl-BIO. This allows an application to handle an SSL/TLS connection using mostly the same API calls as a plain TCP connection (using a socket-BIO) or a local file, as well as the provided 'filter' BIOs like a digest (md) BIO or a base64 encoding/decoding BIO, and any additional BIOs you add. This is the case for the IBM webpage you linked (which is for a client not a server BTW). This is similar to the Unix 'everything is (mostly) a file' philosophy, where for example the utility program grep, by simply calling read on fd 0, can search data from a file, the terminal, a pipe from another program, or (if run under inetd or similar) from a remote system using TCP (but not SSL/TLS, because that isn't in the OS). I haven't encountered many cases where it is particularly beneficial to be able to easily interchange SSL/TLS data with some other type of source/sink, but OpenSSL does provide the ability.
I have a single-threaded client/server application that needs to do both encryption and decryption of their network communication. I plan on using OpenSSL's EVP API and AES-256-CBC.
Some sample pseudo-code I found from a few examples:
// key is 256 bits (32 bytes) when using EVP_aes_256_*()
// I think iv is the same size as the block size, 128 bits (16 bytes)...is it?
1: EVP_CIPHER_CTX *ctx = EVP_CIPHER_CTX_new();
2: EVP_CipherInit_ex(ctx, EVP_aes_256_cbc(), NULL, key, iv, 1); //0=decrypt, 1=encrypt
3: EVP_CipherUpdate(ctx, outbuf, &outlen, inbuf, inlen);
4: EVP_CipherFinal_ex(ctx, outbuf + outlen, &tmplen));
5: outlen += tmplen;
6: EVP_CIPHER_CTX_cleanup(ctx);
7: EVP_CIPHER_CTX_free(ctx);
The problem is from all these examples, I'm not sure what needs to be done at every encryption/decryption, and what I should only do once on startup.
Specifically:
At line 1, do I create this EVP_CIPHER_CTX just once and keep re-using it until the application ends?
Also at line 1, can I re-use the same EVP_CIPHER_CTX for both encryption and decryption, or am I supposed to create 2 of them?
At line 2, should the IV be re-set at every packet I'm encrypting? Or do I set the IV just once, and then let it continue forever?
What if I'm encrypting UDP packets, where a packet can easily go missing or be received out-of-order: am I correct in thinking CBC wont work, or is this where I need to reset the IV at the start of every packet I send out?
Sorry for reviving an old thread, but I noticed the following error in the accepted answer:
At line 1, do I create this EVP_CIPHER_CTX just once and keep re-using it until the application ends?
You create it once per use. That is, as you need to encrypt, you use the same context. If you need to encrypt a second stream, you would use a second context. If you needed to decrypt a third stream, you would use a third context.
Also at line 1, can I re-use the same EVP_CIPHER_CTX for both encryption and decryption, or am I supposed to create 2 of them?
No, see above.
This is not necessary. From the man page for OpenSSL:
New code should use EVP_EncryptInit_ex(), EVP_EncryptFinal_ex(), EVP_DecryptInit_ex(), EVP_DecryptFinal_ex(),
EVP_CipherInit_ex() and EVP_CipherFinal_ex() because they can reuse an existing context without allocating and freeing it up on each call.
In other words, you need to re-initialize the context each time before you use it, but you can certainly use the same context over and over again without creating (allocating) a new one.
I have a single-threaded client/server application that needs to do both encryption and decryption of their network communication. I plan on using OpenSSL's EVP API and AES-256-CBC.
If you are using the SSL_* functions from libssl, then you will likely never touch the EVP_* APIs.
At line 1, do I create this EVP_CIPHER_CTX just once and keep re-using it until the application ends?
You create it once per use. That is, as you need to encrypt, you use the same context. If you need to encrypt a second stream, you would use a second context. If you needed to decrypt a third stream, you would use a third context.
Also at line 1, can I re-use the same EVP_CIPHER_CTX for both encryption and decryption, or am I supposed to create 2 of them?
No, see above.
The ciphers will have different states.
At line 2, should the IV be re-set at every packet I'm encrypting? Or do I set the IV just once, and then let it continue forever?
No. You set the IV once and then forget about it. That's part of the state the context object manages for the cipher.
What if I'm encrypting UDP packets, where a packet can easily go missing or be received out-of-order: am I correct in thinking CBC wont work...
If you are using UDP, its up to you to detect these sorts of problems. You'll probably end up reinventing TCP.
Encryption alone is usually not enough. You also need to ensure authenticity and integrity. You don't operate on data that's not authentic. That's what keeps getting SST/TLS and SSH in trouble.
For example, here's the guy who wrote the seminal paper on authenticated encryption with respect to IPSec, SSL/TLS and SSH weighing in on the Authenticate-Then-Encrypt (EtA) scheme used by SSL/TLS: Last Call: (Encrypt-then-MAC for TLS and DTLS) to Proposed Standard:
The technical results in my 2001 paper are correct but the conclusion
regarding SSL/TLS is wrong. I assumed that TLS was using fresh IVs and
that the MAC was computed on the encoded plaintext, i.e.
Encode-Mac-Encrypt while TLS is doing Mac-Encode-Encrypt which is
exactly what my theoretical example shows is insecure.
For authenticity, you should forgo CBC mode and switch to GCM mode. GCM is an authenticated encryption mode, and it combines confidentiality and authenticity into one mode so you don't have to combine primitives (like AES/CBC with an HMAC).
or is this where I need to reset the IV at the start of every packet I send out?
No, you set the IV once and then forget about it.
The problem is from all these examples, I'm not sure what needs to be done at every encryption/decryption, and what I should only do once on startup.
Create this once: EVP_CIPHER_CTX
Call this once for setup: EVP_CipherInit
Call this as many times as you'd like: EVP_CipherUpdate
Call this once for cleanup: EVP_CipherFinal
The OpenSSL wiki has quite a few examples of using the EVP_* interfaces. See EVP Symmetric Encryption and Decryption, EVP Authenticated Encryption and Decryption and EVP Signing and Verifying.
All the examples use the same pattern: Init, Update and then Final. It does not matter if its encryption or hashing.
Related: this should be of interest to you: EVP Authenticated Encryption and Decryption. Its sample code from the OpenSSL wiki.
Related: you can find copies of Viega, Messier and Chandra's Network Security with OpenSSL online. You might consider hunting down a copy and getting familiar with some of its concepts.
Sorry for reviving an old thread too, but LazerSharks asked twice about evp cipher context in comments. I don't have enough reputation points here to add some comments that's why I'll take an answer here. (Google search even now doesn't show needful information)
From the "Network Security with OpenSSL" book by Pravir Chandra, Matt Messier, John Viega:
Before we can begin encrypting or decrypting, we must allocate and initialize a cipher context. The cipher context is a data structure that keeps track of all relevant state for the purposes of encrypting or decrypting data over a period of time. For example, we can have multiple streams of
data encrypted in CBC mode. The cipher context will keep track of the key associated with each
stream and the internal state that needs to be kept between messages for CBC mode. Additionally, when encrypting with a block-based cipher mode, the context object buffers data that doesn't exactly align to the block size until more data arrives, or until the buffer is explicitly flushed, at which point the data is usually padded as appropriate.
The generic cipher context type is EVP_CIPHER_CTX. We can initialize one, whether it was allocated dynamically or statically, by calling EVP_CIPHER_CTX_init, like so:
EVP_CIPHER_CTX *x = (EVP_CIPHER_CTX *)malloc(sizeof(EVP_CIPHER_CTX));
EVP_CIPHER_CTX_init(x);
Just to complement bacchuswng's answer I have been recently experimenting with LibreSSL and it seems there is no problem with reusing an EVP_CIPHER_CTX but you need to make sure you call EVP_CIPHER_CTX_reset or EVP_CIPHER_CTX_cleanup before starting another encryption / decryption. This is because EVP_EncryptInit / EVP_DecryptInit will clear the context's memory effectively leaking the context's cipher_data previously being used.
Some ciphers don't need to allocate cipher_data but the one I was testing (EVP_aes_128_gcm) does need 680 bytes which can grow out of control quite rapidly.
I can't tell if this is the same behavior with OpenSSL but since documentation for this library is way too hard to come by, I figured I'd share this little quirk (bug perhaps?).
I developed an application in C++ using Crypto++ to encrypt information and store the file in the hard drive. I use an integrity string to check if the password entered by the user is correct. Can you please tell me if the implementation generates a secure file? I am new to the world of the cryptography and I made this program with what I read.
string integrity = "ImGood"
string plaintext = integrity + string("some text");
byte password[pswd.length()]; // The password is filled somewhere else
byte salt[SALT_SIZE]; // SALT_SIZE is 32
byte key[CryptoPP::AES::MAX_KEYLENGTH];
byte iv[CryptoPP::AES::BLOCKSIZE];
CryptoPP::AutoSeededRandomPool rnd;
rnd.GenerateBlock(iv, CryptoPP::AES::BLOCKSIZE);
rnd.GenerateBlock(salt, SALT_SIZE);
CryptoPP::PKCS5_PBKDF2_HMAC<CryptoPP::SHA512> gen;
gen.DeriveKey(key, CryptoPP::AES::MAX_KEYLENGTH, 32,
password, pswd.length(),
salt, SALT_SIZE,
256);
CryptoPP::StringSink* sink = new CryptoPP::StringSink(cipher);
CryptoPP::Base64Encoder* base64_enc = new CryptoPP::Base64Encoder(sink);
CryptoPP::CFB_Mode<CryptoPP::AES>::Encryption cfb_encryption(key, CryptoPP::AES::MAX_KEYLENGTH, iv);
CryptoPP::StreamTransformationFilter* aes_enc = new CryptoPP::StreamTransformationFilter(cfb_encryption, base64_enc);
CryptoPP::StringSource source(plaintext, true, aes_enc);
sstream out;
out << iv << salt << cipher;
The information in the string stream "out" is then written to a file. Another thing is that I don't know what the "purpose" parameter in the derivation function means, I'm guessing it is the desired length of the key so I put 32, but I'm not sure and I can't find anything about it in the Crypto++ manual.
Any opinion, suggestion or mistake pointed out is appreciated.
Thank you very much in advance.
A file can be "secure" only if you define what you mean by "secure".
Usually, you will be interested in two properties:
Confidentiality: the data that is encrypted shall remain unreadable to attackers; revealing the plaintext data requires knowledge of a specific secret.
Integrity: any alteration of the data should be reliably detected; attackers shall not be able to modify the data in any way (even "blindly") without the modification being noticed by whoever decrypts the data.
Your piece of code, apparently, fulfils confidentiality (to some extent) but not integrity. Your string called "integrity" is a misnomer: it is not an integrity check. Its role is apparently to detect accidental password mistakes, not attacks; thus, it would be less confusing if that string was called passwordVerifier instead. An attacker can alter any bit beyond the first 48 bits without the decryption process noticing anything.
Adding integrity (the genuine thing) requires the use of a MAC. Combining encryption and a MAC securely is subject to subtleties; therefore, it is recommended to use for encryption and MAC an authenticated encryption mode which does both, and does so securely (i.e. that specific combination was explicitly reviewed by hordes of cryptographers). Usual recommended AE modes include GCM and EAX.
An important point to note is that, in a context where integrity matters, data cannot be processed before having been verified. This has implications for big files: if your huge file is adorned with a single MAC (whether "manually" or as part of an AE mode), then you must first verify the complete file before beginning to do anything with the plaintext data. This does not work well with streamed processing (e.g. if playing a huge video). A workaround is to split the data into individual chunks, each with its own MAC, but then some care must be taken about the ordering of chunks (attackers could try to remove, duplicate or reorder chunks): things become complex. Complexity, on a general basis, is bad for security.
There are contexts where integrity does not matter. For instance, if your attack model is "the attacker steals the laptop", then you only have to care about confidentiality. However, if the attack model is "the attacker steals the laptop, modifies a few files, and puts it back in my suitcase without me noticing", then integrity matters: the attacker could plant a modification in the file, and infer parts of the secret itself based on your external behaviour when you next access the file.
For confidentiality, you use CFB, which is a bit old-style, but not wrong. For the password-to-key transform, you use PBKDF2, which is fine; the iteration count, though, is quite low: you use 256. Typical values are 20000 or more. The theory is that you should make actual performance measures to set this count to as high a value as you can tolerate: a higher value means slower processing, both for you and for the attacker, so you ought to crank that up (depending on your patience).
Mandatory warning: you are in the process of defining your own crypto, which is a path fraught with perils. Most people who do that produce weak systems, and that includes trained cryptographers; in fact, being a trained cryptographer does not mean that you know how to define a secure protocol, but rather that you know better than defining your own protocol. You are thus highly encouraged to rely on an existing protocol (or format), rather than making your own. I suggest OpenPGP, with (for instance) GnuPG as support library. Even if you need for some reason (e.g. licence issues) to reimplement the format, using a standard format is still a good idea: it avoids introducing weaknesses, it promotes interoperability, and it can be tested against existing systems.
EDIT : My original title was "Use of Stub in RPC" ; I edited the title just to let others know it is more than that question.
I have started developing some SOAP based services and I cannot understand the role of stubs. To quote Wiki :
The client and server use different address spaces, so conversion of parameters used in a function call have to be performed, otherwise the values of those parameters could not be used, because of pointers to the computer's memory pointing to different data on each machine. The client and server may also use different data representations even for simple parameters (e.g., big-endian versus little-endian for integers.) Stubs are used to perform the conversion of the parameters, so a Remote Function Call looks like a local function call for the remote computer.
This is dumb, but I don't understand this "practically". I have done some socket programming in Java, but I don't remember any step for "conversion of parameters" when my TCP/UDP clients interacted with my server. (I assume raw server-client communication using TCP/UDP sockets does come under RPC)
I have had some experience with RESTful service development, but I can't recognize the Stub analogue with REST either. Can someone please help me ?
Stubs for calls over the network (be they SOAP, REST, CORBA, DCOM, JSON-RPC, or whatever) are just helper classes that give you a wrapper function that takes care of all the underlying details, such as:
Initializing your TCP/UDP/whatever transport layer
Finding the right address to call and doing DNS lookups if needed
Connecting to the network endpoint where the server should be
Handling errors if the server isn't listening
Checking that the server is what we're expecting it to be (security checks, versioning, etc)
Negotiating the encoding format
Encoding (or "marshalling") your request parameters in a format suitable for transmission on the network (CDR, NDR, JSON, XML, etc.)
Transmitting your encoded request parameters over the network, taking care of chunking or flow control as necessary
Receiving the response(s) from the server
Decoding (or "unmarshalling") the response details
Returning the responses to your original calling code (or throwing an error if something went wrong)
There's no such thing as "raw" TCP communication. If you are using it in a request/response model and infer any kind of meaning from the data sent across the TCP connection then you've encoded some form of "parameters" in there. You just happened to build yourself what stubs would normally have provided.
Stubs try to make your remote calls look just like local in-process calls, but honestly that's a really bad thing to do. They're not the same at all, and they should be considered differently by your application.