I need to inspect and modify the IPv6 extension headers. So I set up a raw socket to listen for all IP packets on the local address.
package main
import (
"log"
"net"
)
func main() {
c, err := net.ListenIP("ip6:tcp", &net.IPAddr{
IP: net.IPv6loopback,
Zone: "",
})
if err != nil {
panic(err)
}
buf := make([]byte, 1024)
for {
numRead, ipaddr, err := c.ReadFromIP(buf)
log.Print(numRead, ipaddr, err)
log.Printf("% X\n", buf[:numRead])
}
}
I tried all the Read*() methods on the connection but it seems like they just return the payload without the header.
So my question is: How can I access the IPv6 header of a packet?
Raw sockets with IPv6 are different compared to IPv4. Relevant for your case is RFC 3542. Note:
Another difference from IPv4 raw sockets is that complete packets (that is, IPv6 packets with extension headers) cannot be sent or received using the IPv6 raw sockets API. Instead, ancillary data objects are used to transfer the extension headers and hoplimit information, as described in Section 6. Should an application need access to the complete IPv6 packet, some other technique, such as the datalink interfaces BPF or DLPI, must be used.
You can find this out on your own. By running (and strace'ing) your program on my box for every packet I get:
recvfrom(3, "\x45\x00\x00\x3c\x6a\x7d...
^^
This means IP version = 4, IHL = 5 (20 bytes).
When I try the same thing with IPv6 (i.e. your original code), I get something different every time:
recvfrom(3, "\xc6\x22\x00\x50\x4d
In this case every time the kernel is returning stuff starting directly with TCP (in this case the port is 50722, i.e 0xC622).
Another interesting part should be noted in the sources:
switch sa := sa.(type) {
case *syscall.SockaddrInet4:
addr = &IPAddr{IP: sa.Addr[0:]}
n = stripIPv4Header(n, b)
case *syscall.SockaddrInet6:
addr = &IPAddr{IP: sa.Addr[0:], Zone: zoneToString(int(sa.ZoneId))}
}
The header is stripped manually for IPv4 but not for IPv6: for IPv6 there is nothing to remove.
Note, there are mechanisms that will return extra information (but by no means the whole packet). For example the IPV6_RECVPKTINFO socket option will give you access to:
struct in6_pktinfo {
struct in6_addr ipi6_addr; /* src/dst IPv6 address */
unsigned int ipi6_ifindex; /* send/recv interface index */
};
Similar options exist for the routing header option, hop limit etc.
Related
NOTE
I rewrote this question for the bounty as i was able to figure out how to get the first question solved but didn't want to start a new question. the comments below pertain to the original question, not the revised.
Description of Issue
My tcp client executes querying the tcp server one time successfully, returns the response, and then subsequent requests to the server from the client fails. Also, if i terminate the client and reload it fails on the first attempt as well.
Here's what my command prompt looks like:
root#ubuntuT:/home/jon/gocode/udps# ./udpservtcpclient
Text to send: SampleQuery
Message from server: SampleResponse
Text to send: SampleQuery
((End - No Response))
root#ubuntuT:/home/jon/gocode/udps# ./udpservtcpclient
Text to send: SampleQuery
((End - No Response))
What I expect
I expect to be able to query the tcp server from the tcp client endlessly, and have the tcp server return a response from the UDP Server every time. Also, if i terminate the tcp client and reload it should also query correctly to the tcp server without hiccups.
What I think
Something is incorrect with the tcp server accepting connections. I moved the UDP Portion in separate code (not here as not important still does not work) to its own function which opens and closes UDP connections and it still does not function after first connection.
UPDATE
I updated the code to display "Accepted Connection" just below c,err:=serverConn.Accept() and it only printed once, for the first request. any subsequent queries from the client did not display the line so it has to do with accepting connections
Sourcecode
Server Code:
package main
import (
"log"
//"fmt"
"net"
//"strings"
"bufio"
//"time"
//"github.com/davecgh/go-spew/spew"
)
var connUDP *net.UDPConn
func udpRoutine(query string)(string){
connUDP.Write([]byte(query))
buf:= make([]byte,128)
n,err:=connUDP.Read(buf[0:])
if err != nil{
log.Fatal(err)
}
response := string(buf[0:n])
return response
}
func handle(conn net.Conn) error {
defer func(){
conn.Close()
}()
r := bufio.NewReader(conn)
w := bufio.NewWriter(conn)
scanr := bufio.NewScanner(r)
for {
scanned := scanr.Scan()
if !scanned {
if err := scanr.Err(); err != nil {
log.Printf("%v(%v)", err, conn.RemoteAddr())
return err
}
break
}
response:=udpRoutine(scanr.Text())
w.WriteString(response+"\n")
w.Flush()
}
return nil
}
func main(){
// setup tcp listener
serverConn,err := net.Listen("tcp","127.0.0.1:8085")
if err != nil{
log.Fatal(err)
}
defer serverConn.Close()
// setup udp client
udpAddr,err:=net.ResolveUDPAddr("udp4","127.0.0.1:1175")
if err != nil{
log.Fatal(err)
}
connUDP,err=net.DialUDP("udp",nil,udpAddr)
if err != nil{
log.Fatal(err)
}
defer connUDP.Close()
for{
c,err:=serverConn.Accept()
if err != nil{
log.Fatal(err)
}
//c.SetDeadline(time.Now().Add(5))
go handle(c)
}
}
Client Code:
package main
import "net"
import "fmt"
import "bufio"
import "os"
func main() {
// connect to this socket
conn, _ := net.Dial("tcp", "127.0.0.1:8085")
for {
reader := bufio.NewReader(os.Stdin)
// read in input from stdin
fmt.Print("Text to send: ")
text,_ := reader.ReadString('\n')
// send to socket
fmt.Fprintf(conn, text + "\n")
// listen for reply
message, _ := bufio.NewReader(conn).ReadString('\n')
fmt.Print("Message from server: "+message)
}
}
It seems there are two problems here:
1) UDP Server
Your question describes an issue when the client is not able to make a second request to the server.
I used a simple echo UDP server, along with the code you posted for the server and client and can't reproduce the problem (I can still make several requests to the server), so I suspect that has to do with the UDP server you are using (code for which I can't see in this question).
I suggest you try this with a simple UDP server that just echoes messages back:
package main
import (
"fmt"
"net"
)
func main() {
conn, _ := net.ListenUDP("udp", &net.UDPAddr{IP:[]byte{0,0,0,0},Port:1175,Zone:""})
defer conn.Close()
buf := make([]byte, 1024)
for {
n, addr, _ := conn.ReadFromUDP(buf)
conn.WriteTo(buf[0:n], addr)
fmt.Println("Received ", string(buf[0:n]), " from ", addr)
}
}
2) Extra new line in TCP Client
Using the exact code you posted and that UDP server I posted above, this seems to work, but the output I get on the client is not what I would have expected.
It seems that is cause by a second issue which is this line in the client:
// send to socket
fmt.Fprintf(conn, text + "\n")
That line end you are sending is causing the scanner you use on the server to read two lines (the text you send and then an empty line), making the server write two lines back to the client.
But in the client you only read one line, so the second line seems to be pending until the client connects again.
That can be fixed by simply changing that to:
// send to socket
fmt.Fprintf(conn, text)
Output for the fixed code
Using that UDP server and making that change to the client, this is the output I get when running all three components:
Text to send: first msg
Message from server: first msg
Text to send: second msg
Message from server: second msg
Text to send: third msg
Message from server: third msg
Text to send:
I can then stop just the client, start it again and it'd keep working:
Text to send: fourth msg
Message from server: fourth msg
Text to send:
Aditional notes
About the two other lines in the client code that use newlines:
// read in input from stdin
fmt.Print("Text to send: ")
text,_ := reader.ReadString('\n')
That newline is needed there, cause when you input the text using standard input, you finish the line using the enter key (and thus ending the line with a newline), so the line should be read until the \n character.
message, _ := bufio.NewReader(conn).ReadString('\n')
That one is needed cause when the server writes the response to the connection it does w.WriteString(response+"\n"). So the response includes a newline at the end, and you should read up to that newline when reading the response text.
I'm trying to get the socket id/fd of a net/http request so that i can use it with syscall.Bind() to bind the socket to one of my many public outgoing IPV4 addresses.
I want to be able to select which IP address is used for the outgoing request. This is for Windows.
Any help is greatly appreciated.
Below is some code which is for linux, but i need to get the http.Client's socket fd and not net.Conn.
func bindToIf(conn net.Conn, interfaceName string) {
ptrVal := reflect.ValueOf(conn)
val := reflect.Indirect(ptrVal)
//next line will get you the net.netFD
fdmember := val.FieldByName("fd")
val1 := reflect.Indirect(fdmember)
netFdPtr := val1.FieldByName("sysfd")
fd := int(netFdPtr.Int())
//fd now has the actual fd for the socket
err := syscall.SetsockoptString(fd, syscall.SOL_SOCKET,
syscall.SO_BINDTODEVICE, interfaceName)
if err != nil {
log.Fatal(err)
}
}
I'm trying to get the socket id/fd of a net/http request
Neither http.Request or http.Client have a socket for you to get.
You can customize how an http.Client creates TCP connections by modifying it's Transport. See the Dial and DialTLS functions.
From the docs:
Dial specifies the dial function for creating unencrypted TCP connections. If Dial is nil, net.Dial is used.
You may be interested in this question, which asks how to dial using a specific interface.
You could set up the default transport do something like this:
http.DefaultTransport.(*http.Transport).Dial = func(network, addr string) (net.Conn, error) {
d := net.Dialer{LocalAddr: /* your addr here */}
return d.Dial(network, addr)
}
If you're using TLS, you'll want to do something similar for http.DefaultTransport.DialTLS.
Suppose I had a Tcp server in linux, it would create a new goroutine for a new connnection. When I want to write data to the tcp connection, should I do it just like this
conn.Write(data)
or do it in a goroutine especially for writing, like this
func writeRoutine(sendChan chan []byte){
for {
select {
case msg := <- sendChan :
conn.Write(msg)
}
}
}
just in case that the network was busy.
In a short, Did I need a write buffer in go just like in c/c++ when writing to a socket?
PS maybe I didn't exclaim the problem clearly.
1 I talked of the server, meaning a tcp server runing in linux. It would create a new goroutine for a new connnection. like this
listener, err := net.ListenTCP("tcp", tcpAddr)
if err != nil {
log.Error(err.Error())
os.Exit(-1)
}
for {
conn, err := listener.AcceptTCP()
if err != nil {
continue
}
log.Debug("Accept a new connection ", conn.RemoteAddr())
go handleClient(conn)
}
2 I think my problem isn't much concerned with the code. As we know, when we use size_t write(int fd, const void *buf, size_t count); to write a socket fd in c/c++, for a tcp server, we need a write buffer for a socket in your code necessaryly, or maybe only some of the data is writen successfully. I mean, Do I have to do so in go ?
You are actually asking two different questions here:
1) Should you use a goroutine per accepted client connection in my TCP server?
2) Given a []byte, how should I write to the connection?
For 1), the answer is yes. This is the type of pattern that go is most suited for. If you take a look at the source code for the net/http, you will see that it spawns a goroutine for each connection.
As for 2), you should do the same that you would do in a c/c++ server: write, check how much was written and keep on writing until your done, always checking for errors. Here is a code snippet on how to do it:
func writeConn(data []byte) error {
var start,c int
var err error
for {
if c, err = conn.Write(data[start:]); err != nil {
return err
}
start += c
if c == 0 || start == len(data) {
break
}
}
return nil
}
server [...] create a new goroutine for a new connnection
This makes sense because the handler goroutines can block without delaying the server's accept loop.
If you handled each request serially, any blocking syscall would essentially lock up the server for all clients.
goroutine especially for writing
This would only make sense in use cases where you're writing either a really big chunk of data or to a very slow connection and you need your handler to continue unblocked, for instance.
Note that this is not what is commonly understood as a "write buffer".
I have two network links to the Internet, and I have two default routes set up:
Destination Gateway Genmask Flags Metric Ref Use Iface
default gateway0 0.0.0.0 UG 0 0 eth0
default gateway1 0.0.0.0 UG 0 0 eth1
...
I created two sockets with BINDTODEVICE, so that I can send data out either eth0 or eth1. I am also trying to listen on both sockets using recvfrom (UDP data only), but I can only successfully read data from whichever interface is listed first in the routes. eth0 works, for example, but I get nothing from the socket bound to eth1.
Running wireshark on either interface shows data coming in successfully - that is, I can see data being sent from the Internet to either eth0's or eth1's IP in Wireshark (so NAT is not a problem with either), but my program just blocks on recvfrom without getting any data.
I have tried using bind on the sockets to make them listen on their respective interface's IP, and also tried not using bind to have them listen on 0.0.0.0 (each on a different port), but I still have the same problem.
How can I make sure both sockets get the data they're supposed to?
Edit: sample code:
int createDeviceBoundUDPSocket(uint32_t sip, uint16_t sport, const char* bind_dev) {
printf("bind_dev = %s", bind_dev);
int s = socket(AF_INET, SOCK_DGRAM, 0);
int result;
struct sockaddr_in my_ip_addr;
if (s < 0) {
perror("socket");
return s;
}
memset(&my_ip_addr, 0, sizeof(my_ip_addr));
my_ip_addr.sin_family = AF_INET;
my_ip_addr.sin_addr.s_addr = htonl(sip);
my_ip_addr.sin_port = htons(sport);
// commenting this section out doesn't seem to make a difference
// listening on 0.0.0.0 or the interface's IP both have the same problem
result = bind(s, (struct sockaddr*)(&my_ip_addr), sizeof(my_ip_addr));
if (result < 0) {
perror("Error in bind");
return result;
}
if (bind_dev) {
// Bind to specific device.
if (setsockopt(s, SOL_SOCKET, SO_BINDTODEVICE,
bind_dev, strlen(bind_dev) + 1)) {
perror("Error binding to device");
return -1;
}
}
return s;
}
The solution, as hinted by the Linux Advanced Routing and Traffic Control guide, is to turn off reverse path filtering. Not sure which of the three interfaces needed to have it turned off, but doing
echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
made it work.
let me first tell what I am trying to do.
I am trying to write a very simple proxy server.
I used the socket API to create a socket.
socket = socket(AF_INET, SOCK_STREAM, 0));
my proxy server worked fine until I tried it for a streaming data.
So what I did was my server socket listened to the requests and parsed them and then forwarded them to the actual server, I then used the read() call to read the packet & I blindly forward it back to the client.
For all html pages and images it works fine. but when I try to forward a streaming video I am not able to do it.
My socket always returns the application layer data (HTTP packet) but in a streaming video only the first packet is http and rest all are just TCP packets. So I am able to forward only the first HTTP packet. When I try to read the other packets which contain data (which are all TCP) I don't get anything at the application layer (which is obvious as there is nothing at application layer in those packets ). So I am stuck and I do not know how to read those packets from TCP layer (I dont wanna use raw socket) and get my job done.
thanks in advance
You have to parse the packet header to know how much data to read from the socket. at first, use a ring buffer (a circular one!) for example the BSD sys/queue.h to order the received data from the stream.
The code below shows how to extract header_length, total_length, source and destination Address of an IPv4 packet in layer 3. refer to IPv4 packet layout to understand offsets:
typedef struct {
unsigned char version;
unsigned char header_length;
unsigned short total_length;
struct in_addr src;
struct in_addr dst;
} Packet;
int rb_packet_write_out(RingBuffer *b, int fd, int count) {
int i;
for (i = 0; i < count; i++) {
if (b->level < 20) {
return i;
}
Packet p;
unsigned char *start = b->blob + b->read_cursor;
unsigned char b1 = start[0];
p.version = b1 >> 4;
p.header_length = b1 & 0xf;
p.total_length = bigendian_deserialize_uint16(start + 2);
if (b->level < p.total_length) {
return i;
}
memcpy(&(p.src), start + 12, 4);
memcpy(&(p.dst), start + 16, 4);
char s[5], d[5];
inet_ntop(AF_INET, &(p.src), s, INET_ADDRSTRLEN);
inet_ntop(AF_INET, &(p.dst), d, INET_ADDRSTRLEN);
L_DEBUG("Packet: v%u %s -> %s (%u)", p.version, s, d, p.total_length);
}
return i;
}
If you use the socket API, then you are on the layer below HTTP, that is, to you everything is "just TCP". If the connection is stuck somewhere, it is most likely that something else is broken. Note there is no guarantee that the HTTP request or reply header will even fit in a single packet; they just usually do.
An HTTP 1.1 compliant streaming server will use "Content-Encoding: chunked" and report the length of each chunk rather than the length of the entire file, you should keep that in mind when proxying.
So what I did was my server socket
listened to the requests and parsed
them
Why? An HTTP proxy doesn't have to parse anything except the first line of the request, to know where to make the upstream connection to. Everything else is just copying bytes in both directions.