I'm trying to make a very simple http server with the low-level Unix module
to get a better understanding of how a http server actually works.
This is my current code:
let () =
let socket_fd = Unix.socket Unix.PF_INET Unix.SOCK_STREAM 0 in
let _ = Unix.bind socket_fd (Unix.ADDR_INET ((Unix.inet_addr_of_string "127.0.0.1"), 8888)) in
let _ = Unix.listen socket_fd 1 in
let (file_descr,sockaddr) = Unix.accept socket_fd in
let b = Bytes.create 1024 in
let _ = Unix.recv file_descr b 0 0 [Unix.MSG_PEEK] in
print_endline b;
When I open a webbrowser & surf to localhost:8888 the program displays a
newline and exits. Why isn't it printing out the request?
According to the Unix module documentation recv has the following signature:
val recv : file_descr -> bytes -> int -> int -> msg_flag list -> int
Receive data from a connected socket.
What is the meaning of the ... -> int -> int -> ... ? I have no idea what I should be passing in those two ints.
Also how do I know beforehand what size a should give to bytes ?
Part of the answer lies in the C mapping of recv (sendrecv: .opam/version/build/ocaml/otherlibs/unix/sendrecv.c ).
CAMLprim value unix_recv(value sock, value buff, value ofs, value len,
value flags)
{
int ret, cv_flags;
long numbytes;
char iobuf[UNIX_BUFFER_SIZE];
cv_flags = convert_flag_list(flags, msg_flag_table);
Begin_root (buff);
numbytes = Long_val(len);
if (numbytes > UNIX_BUFFER_SIZE) numbytes = UNIX_BUFFER_SIZE;
enter_blocking_section();
ret = recv(Int_val(sock), iobuf, (int) numbytes, cv_flags);
leave_blocking_section();
if (ret == -1) uerror("recv", Nothing);
memmove (&Byte(buff, Long_val(ofs)), iobuf, ret);
End_roots();
return Val_int(ret);
}
So first int is an offset within the buffer, and second int is the length of the retrieval - that you shoud not set to 0.
Happy new year anyway ;)
Related
I ask for your help to understand part of the specification of the FastCGI protocol.
Currently this is the code I have:
#![allow(non_snake_case)]
#![allow(unused_must_use)]
use std::os::unix::net::{UnixStream};
use std::io::{Read, Write};
fn main() {
pub const FCGI_VERSION_1: u8 = 1;
pub const FCGI_BEGIN_REQUEST:u8 = 1;
pub const FCGI_RESPONDER: u16 = 1;
pub const FCGI_PARAMS: &str = "FCGI_PARAMS";
let socket_path = "/run/php-fpm/php-fpm.sock";
let mut socket = match UnixStream::connect(socket_path) {
Ok(sock) => sock,
Err(e) => {
println!("Couldn't connect: {e:?}");
return
}
};
let requestId: u16 = 1;
let role: u16 = FCGI_RESPONDER;
let beginRequest = vec![
// FCGI_Header
FCGI_VERSION_1, FCGI_BEGIN_REQUEST,
(requestId >> 8) as u8, (requestId & 0xFF) as u8,
0x00, 0x08, // This is the size of `FCGI_BeginRequestBody`
0, 0,
// FCGI_BeginRequestBody
(role >> 8) as u8, (role & 0xFF) as u8,
0, // Flags
0, 0, 0, 0, 0, // Reserved
];
socket.write_all(&beginRequest).unwrap();
let data = vec![
(100) as u8, // this value is just an example
];
let contentLength = data.len();
assert!(contentLength <= usize::MAX);
let requestHeader = vec![
FCGI_VERSION_1, FCGI_BEGIN_REQUEST,
(requestId >> 8) as u8, (requestId & 0xFF) as u8,
(contentLength >> 8) as u8, (contentLength & 0xFF) as u8,
0, 0,
];
socket.write_all(&requestHeader).unwrap();
}
I have this code thanks to the answer of my last question related to this topic, so, with that example code (which works perfectly for me) I would like to ask you my question.
How can I write the FCGI_PARAMS?
I mean, if I understand correctly, the documentation says:
FCGI_PARAMS is a stream record type used in sending name-value pairs from the Web server to the application
This means that the FCGI_PARAMS are Name-Value Pairs. And the part of the documentation that describes the Name-Value Pairs says:
FastCGI transmits a name-value pair as the length of the name, followed by the length of the value, followed by the name, followed by the value
Then this way I think that it would be (represented in code):
let param = vec![
"SCRIPT_FILENAME".len(),
"index.php".len(),
"SCRIPT_FILENAME",
"index.php",
]; // it is just an example, but i think it represents what i am talking about
But if I add this code, and then I write it to the socket with the following line:
socket.write_all(¶m);
And then when reading the socket, the socket does not return anything. What am I doing wrong? How should I send the data? I hope you can help me with this, I want to clarify that I am quite new to FastCGI and unix sockets so I am very sorry if any line of my displayed code is poorly exemplified.
Rust doesn't support heterogeneous vectors, so your let param =… shouldn't compile. The way to send the params is to use multiple writes:
let param_name = "SCRIPT_FILENAME".as_bytes(); // could also be written `b"SCRIPT_FILENAME"`
let param_value = "index.php".as_bytes();
let lengths = [ param_name.len() as u8, param_value.len() as u8 ];
socket.write_all (&lengths).unwrap();
socket.write_all (param_name).unwrap();
socket.write_all (param_value).unwrap();
Is there a way to obtain a socket fd from object of type struct sock in the kernel? Quick look inside of struct sock doesn't help to find something that looks like socket descriptor. Basically I need what socket() syscall returns, is it not stored in 'sock' ?
I need to get fd at the point before a packet hits IP stack.
Thanks.
For each process there is a table of file descriptors, which maps file descriptors to struct file objects. You may iterate over this table using iterate_fd() function.
For any struct file you may determine which struct sock object it corresponds using sock_from_file() function.
In total:
/*
* Callback for iterate_fd().
*
* If given file corresponds to the given socket, return fd + 1.
* Otherwise return 0.
*
* Note, that returning 0 is needed for continue the search.
*/
static int check_file_is_sock(void* s, struct file* f, int fd)
{
int err;
struct sock* real_sock = sock_from_file(f, &err);
if(real_sock == s)
return fd + 1;
else
return 0;
}
// Return file descriptor for given socket in given process.
int get_fd_for_sock(struct sock* s, struct task* p)
{
int search_res;
task_lock(p);
// This returns either (fd + 1) or 0 if not found.
search_res = iterate_fd(p->files, 0, &check_file_is_sock, s);
task_unlock(p);
if(search_res)
return search_res - 1;
else
return -1; // Not found
}
I'm creating a TCP client in swift for an app on iOS device. I'm using NSOuputStream to send data on the server.
Our protocol is JSON type and we have to send first the length of the json (unsigned Int 32 bits) and then the JSON content (as UTF8 string).
Every thing work fine, except that I cannot find how to write my Int using write(_:maxLength:) function of NSOutputStream..
To convert my JSON String into UnsafePointer<UInt8> I use the following
//var json: String
let encodedData = [UInt8](json.utf8)
let result = outputStream?.write(encodedData, maxLength: encodedData.count)
Does someone know how to do it for an Int ?
Thanks for your help !
You should be able to do something like this:
var int = 100
let result = withUnsafePointer(&int) {
outputStream?.write(UnsafePointer($0), maxLength: sizeof(Int))
}
EDIT: To read your Int you can do
func getInt(ptr: UnsafePointer<UInt8>) -> Int {
return UnsafePointer(ptr).memory
}
Swift 4:
var int = 100
let result = withUnsafePointer(to: & int) {
let unsafePointer8 = $0.withMemoryRebound(to: UInt8.self, capacity: 1, {$0})
outputStream?.write(unsafePointer8, maxLength: MemoryLayout<Int64>.size)
}
My application uses a somewhat complex inmutable data structure that is encoded in a binary file. I need to have access to it at the byte level, avoiding any copying. Normally, I would use C or C++ pointer arithmetic and typecasts, to access and interpret the raw byte values. I would like to do the same with Swift.
I have found that the following works:
class RawData {
var data: NSData!
init(rawData: NSData) {
data = rawData
}
func read<T>(byteLocation: Int) -> T {
let bytes = data.subdataWithRange(NSMakeRange(byteLocation, sizeof(T))).bytes
return UnsafePointer<T>(bytes).memory
}
func example_ReadAnIntAtByteLocation5() -> Int {
return read(5) as Int
}
}
However, I am not sure how efficient it is. Do data.subdataWithRange and NSMakeRange allocate objects every time I call them, or are they just syntactic sugar for dealing with pointers?
Is there a better way to do this in Swift?
EDIT:
I have created a small Objective-C class that just encapsulates a function to offset a pointer by a given number of bytes:
#implementation RawDataOffsetPointer
inline void* offsetPointer(void* ptr, int bytes){
return (char*)ptr + bytes;
}
#end
If I include this class in the bridging header, then I can change my read method to
func read<T>(byteLocation: Int) -> T {
let ptr = offsetPointer(data.bytes, CInt(byteLocation))
return UnsafePointer<T>(ptr).memory
}
which will not copy data from my buffer, or allocate other objects.
However, it would still be nice to do some pointer arithmetic from Swift, if it were possible.
If you just want to do it directly, UnsafePointer<T> can be manipulated arithmetically:
let oldPointer = UnsafePointer<()>
let newPointer = oldPointer + 10
You can also cast a pointer like so (UnsafePointer<()> is equivalent to void *)
let castPointer = UnsafePointer<MyStruct>(oldPointer)
I would recommend looking into NSInputStream, which allows you to read NSData as a series of bytes (UInt8 in Swift).
Here is a little sample I put together in the playground:
func generateRandomData(count:Int) -> NSData
{
var array = Array<UInt8>(count: count, repeatedValue: 0)
arc4random_buf(&array, UInt(count))
return NSData(bytes: array, length: count)
}
let randomData = generateRandomData(256 * 1024)
let stream = NSInputStream(data: randomData)
stream.open() // IMPORTANT
var readBuffer = Array<UInt8>(count: 16 * 1024, repeatedValue: 0)
var totalBytesRead = 0
while (totalBytesRead < randomData.length)
{
let numberOfBytesRead = stream.read(&readBuffer, maxLength: readBuffer.count)
// Do something with the data
totalBytesRead += numberOfBytesRead
}
You can create an extension to read primitive types like so:
extension NSInputStream
{
func readInt32() -> Int
{
var readBuffer = Array<UInt8>(count:sizeof(Int32), repeatedValue: 0)
var numberOfBytesRead = self.read(&readBuffer, maxLength: readBuffer.count)
return Int(readBuffer[0]) << 24 |
Int(readBuffer[1]) << 16 |
Int(readBuffer[2]) << 8 |
Int(readBuffer[3])
}
}
I would recommend the simple way to use UnsafeArray.
let data = NSData(contentsOfFile: filename)
let ptr = UnsafePointer<UInt8>(data.bytes)
let bytes = UnsafeBufferPointer<UInt8>(start:ptr, count:data.length)
I have an extremely strange bug.
I have two applications that communicate over TCP/IP.
Application A is the server, and application B is the client.
Application A sends a bunch of float values to application B every 100 milliseconds.
The bug is the following: sometimes some of the float values received by application B are not the same as the values transmitted by application A.
Initially, I thought there was a problem with the Ethernet or TCP/IP drivers (some sort of data corruption). I then tested the code in other Windows machines, but the problem persisted.
I then tested the code on Linux (Ubuntu 10.04.1 LTS) and the problem is still there!!!
The values are logged just before they are sent and just after they are received.
The code is pretty straightforward: the message protocol has a 4 byte header like this:
//message header
struct MESSAGE_HEADER {
unsigned short type;
unsigned short length;
};
//orientation message
struct ORIENTATION_MESSAGE : MESSAGE_HEADER
{
float azimuth;
float elevation;
float speed_az;
float speed_elev;
};
//any message
struct MESSAGE : MESSAGE_HEADER {
char buffer[512];
};
//receive specific size of bytes from the socket
static int receive(SOCKET socket, void *buffer, size_t size) {
int r;
do {
r = recv(socket, (char *)buffer, size, 0);
if (r == 0 || r == SOCKET_ERROR) break;
buffer = (char *)buffer + r;
size -= r;
} while (size);
return r;
}
//send specific size of bytes to a socket
static int send(SOCKET socket, const void *buffer, size_t size) {
int r;
do {
r = send(socket, (const char *)buffer, size, 0);
if (r == 0 || r == SOCKET_ERROR) break;
buffer = (char *)buffer + r;
size -= r;
} while (size);
return r;
}
//get message from socket
static bool receive(SOCKET socket, MESSAGE &msg) {
int r = receive(socket, &msg, sizeof(MESSAGE_HEADER));
if (r == SOCKET_ERROR || r == 0) return false;
if (ntohs(msg.length) == 0) return true;
r = receive(socket, msg.buffer, ntohs(msg.length));
if (r == SOCKET_ERROR || r == 0) return false;
return true;
}
//send message
static bool send(SOCKET socket, const MESSAGE &msg) {
int r = send(socket, &msg, ntohs(msg.length) + sizeof(MESSAGE_HEADER));
if (r == SOCKET_ERROR || r == 0) return false;
return true;
}
When I receive the message 'orientation', sometimes the 'azimuth' value is different from the one sent by the server!
Shouldn't the data be the same all the time? doesn't TCP/IP guarantee delivery of the data uncorrupted? could it be that an exception in the math co-processor affects the TCP/IP stack? is it a problem that I receive a small number of bytes first (4 bytes) and then the message body?
EDIT:
The problem is in the endianess swapping routine. The following code swaps the endianess of a specific float around, and then swaps it again and prints the bytes:
#include <iostream>
using namespace std;
float ntohf(float f)
{
float r;
unsigned char *s = (unsigned char *)&f;
unsigned char *d = (unsigned char *)&r;
d[0] = s[3];
d[1] = s[2];
d[2] = s[1];
d[3] = s[0];
return r;
}
int main() {
unsigned long l = 3206974079;
float f1 = (float &)l;
float f2 = ntohf(ntohf(f1));
unsigned char *c1 = (unsigned char *)&f1;
unsigned char *c2 = (unsigned char *)&f2;
printf("%02X %02X %02X %02X\n", c1[0], c1[1], c1[2], c1[3]);
printf("%02X %02X %02X %02X\n", c2[0], c2[1], c2[2], c2[3]);
getchar();
return 0;
}
The output is:
7F 8A 26 BF
7F CA 26 BF
I.e. the float assignment probably normalizes the value, producing a different value from the original.
Any input on this is welcomed.
EDIT2:
Thank you all for your replies. It seems the problem is that the swapped float, when returned via the 'return' statement, is pushed in the CPU's floating point stack. The caller then pops the value from the stack, the value is rounded, but it is the swapped float, and therefore the rounding messes up the value.
TCP tries to deliver unaltered bytes, but unless the machines have similar CPU-s and operating-systems, there's no guarantee that the floating-point representation on one system is identical to that on the other. You need a mechanism for ensuring this such as XDR or Google's protobuf.
You're sending binary data over the network, using implementation-defined padding for the struct layout, so this will only work if you're using the same hardware, OS and compiler for both application A and application B.
If that's ok, though, I can't see anything wrong with your code. One potential issue is that you're using ntohs to extract the length of the message and that length is the total length minus the header length, so you need to make sure you setting it properly. It needs to be done as
msg.length = htons(sizeof(ORIENTATION_MESSAGE) - sizeof(MESSAGE_HEADER));
but you don't show the code that sets up the message...