I want to tunnel a sub-command through a connection by listening to a port, running the sub-command (to connect to that port), and then forwarding the data through the connection:
package main
import (
"fmt"
"net"
"os"
"os/exec"
)
func main() {
ln, err := net.ListenTCP("tcp4", &net.TCPAddr{IP: localhost})
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
defer ln.Close()
port := ln.Addr().(*net.TCPAddr).Port
cmd := exec.Command(
"git",
"clone",
fmt.Sprintf("git://127.0.0.1:%d/project.git", port),
)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
if err := cmd.Start(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
defer cmd.Process.Kill()
errs := make(chan error, 1)
go func() {
errs <- cmd.Wait()
}()
conns := make(chan net.Conn, 1)
go func() {
conn, err := ln.Accept()
if err == nil {
conns <- conn
} else {
fmt.Println(err)
errs <- err
}
}()
select {
case err := <-errs:
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
case conn := <-conns:
defer conn.Close()
// TODO Tunnel data from `conn` through another connection.
}
fmt.Println("done.")
}
var localhost = net.IPv4(127, 0, 0, 1)
However, there's a race here between the time that we start listening and the time when the sub-command actually connects to the listener, where another process can connect to the listener. I believe this race could be exploited by an attacker to communicate with the process at the other end of the connection and achieve results that would otherwise require privilege escalation to perform (example attacks that require special permissions are replacing the git command with a malicious program or simply reading the contents of the cloned directory, in this instance).
Should this be a concern? If so, is there a way it can be prevented? Though the question is asked using Go as an example, answers and comments in any language are welcome.
Yes it is a concern. It can be prevented by using some form of authentication so that your server only allows connections from legitimate clients.
Expanding on Warren Dew's answer, assuming the other end of the tunnel connects to an SSH server then this will perform authentication of the Git server/client.
A more general approach to "add" authentication to a sub-command for this purpose is to wrap the sub-command in a container such as docker and have the sub-command tunnel through a connection that authenticates itself. Though the tunnel's race condition is still present, it is at a higher "level" (i.e. inside the container), meaning that an attacker with no privileges will not be able to exploit the race condition in the base system. The downside to this approach is its inherent complexity and the overhead (though minimal) of running the sub-command in a container.
Related
I want to connect two hosts (libp2p nodes), each of them behind a home router (thus, NAT).
The relay example in the official Go implementation repo simulates two undialable hosts and spawns a relay node, which is used to establish the connection across the NAT. It's a good example, it works and makes sense. I'd like to start from there.
What I can't find is how to automatically discover and use a public relay instead of a dummy one running on the same system.
I asked ChatGPT and she pointed me out to:
func main() {
ctx := context.Background()
// Create the first node
node1, err := libp2p.New(ctx)
if err != nil {
panic(err)
}
// Create the second node
node2, err := libp2p.New(ctx)
if err != nil {
panic(err)
}
// Set up a relay for each node
err = relay.NewAutoRelay(ctx, node1)
if err != nil {
panic(err)
}
err = relay.NewAutoRelay(ctx, node2)
if err != nil {
panic(err)
}
But it doesn't sound sound at all.
What's the standard setup to make two nodes behind NAT talk to each other?
As per the documentation in Readme:
Make sure to defer a call to Disconnect after instantiating your client:
defer func() {
if err = client.Disconnect(ctx); err != nil {
panic(err)
}
}()
Does the above documentation meant to disconnect, during shutdown of the program(using mongodb driver)?
They just reminded you that you should always close the connection to the database at some point. When exactly is up to you. Usually, you initialize the database connection at the top-level of your application, so the defer call should be at the same level. For example,
func main() {
client, err := mongo.Connect(ctx, options.Client().ApplyURI("mongodb://localhost:27017"))
defer func() {
if err = client.Disconnect(ctx); err != nil {
panic(err)
}
}()
// Pass the connection to other components of your appliaction
doSomeWork(client)
}
Note: If you are using Goroutines don't forget to synchronize them in main otherwise the connection will be close too early.
I think the documentation meant to use defer with client.Disconnect in the main function of your program. Thanks to that your program will close all the MongoDB client connections before exiting.
If you would use it e.g. in a helper function that prepares that client, it would close all the connections right after the client creation which may not be something you want.
How do I create a proper mongo based application in Go using the official driver(go.mongodb.org/mongo-driver/mongo)? I have a MongoConnect() and MongoDisconnect(client) function to create a connection and delete it. But, it's not too efficient and starts leaking FD as the app has got around 40 functions and finding all the missed MongoDisconnect() becomes hectic.
The current MongoConnect and MongoDisconnect are as follows.
func MongoConnect() (*mongo.Client, error) {
clientOptions := options.Client().ApplyURI("mongodb://localhost:27017")
client, err := mongo.Connect(context.TODO(), clientOptions)
if err != nil {
Logger(err)
return nil, err
}
err = client.Ping(context.TODO(), nil)
if err != nil {
Logger(err)
return nil, err
}
return client, err
}
func MongoDisconnect(client *mongo.Client) {
_ = client.Disconnect(context.TODO())
}
I am looking for a method that would still use MongoConnect() to create connections and would automatically kill the client without the usage of MongoDisconnect().
PS. Other methods that are better than the above requirement are also welcome
I'm not sure that there is an 'efficient' way to fix the underlying issue, you probably will need to look at all the places where you've called MongoConnect() and ensure you have a corresponding MongoDisconnect().
In saying that, what you might want to look at is implemententing the right pattern for connecting to databases.
Generally speaking, if your database driver takes care of managing connections for you then you should create the connection once and just pass it around as needed.
You could also defer the closing of that connection to a go routine which would close it once it was no longer needed (when you're application is shutting down).
Here is a code snippet of how this is implemented:
// =========================================================================
// Start Database
log.Println("main: Initializing database support")
db, err := database.Open(database.Config{
User: cfg.DB.User,
Password: cfg.DB.Password,
Host: cfg.DB.Host,
Name: cfg.DB.Name,
DisableTLS: cfg.DB.DisableTLS,
})
if err != nil {
return errors.Wrap(err, "connecting to db")
}
defer func() {
log.Printf("main: Database Stopping : %s", cfg.DB.Host)
db.Close()
}()
I didn't write this code and its part of a larger project scaffold which has some other nice patterns for web applications.
Ultimate service
I have a websocket client. In reality, it is far more complex than the basic code shown below.
I now need to scale this client code to open connections to multiple servers. Ultimately, the tasks that need to be performed when a message is received from the servers is identical.
What would be the best approach to handle this?
As I said above the actual code performed when receiving the message is far more complex than shown in the example.
package main
import (
"flag"
"log"
"net/url"
"os"
"os/signal"
"time"
"github.com/gorilla/websocket"
)
var addr = flag.String("addr", "localhost:1234", "http service address")
func main() {
flag.Parse()
log.SetFlags(0)
interrupt := make(chan os.Signal, 1)
signal.Notify(interrupt, os.Interrupt)
// u := url.URL{Scheme: "ws", Host: *addr, Path: "/echo"}
u := url.URL{Scheme: "ws", Host: *addr, Path: "/"}
log.Printf("connecting to %s", u.String())
c, _, err := websocket.DefaultDialer.Dial(u.String(), nil)
if err != nil {
log.Fatal("dial:", err)
}
defer c.Close()
done := make(chan struct{})
go func() {
defer close(done)
for {
_, message, err := c.ReadMessage()
if err != nil {
log.Println("read:", err)
return
}
log.Printf("recv: %s", message)
}
}()
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
for {
select {
case <-done:
return
case t := <-ticker.C:
err := c.WriteMessage(websocket.TextMessage, []byte(t.String()))
if err != nil {
log.Println("write:", err)
return
}
case <-interrupt:
log.Println("interrupt")
// Cleanly close the connection by sending a close message and then
// waiting (with timeout) for the server to close the connection.
err := c.WriteMessage(websocket.CloseMessage, websocket.FormatCloseMessage(websocket.CloseNormalClosure, ""))
if err != nil {
log.Println("write close:", err)
return
}
select {
case <-done:
case <-time.After(time.Second):
}
return
}
}
}
Modify the interrupt handling to close a channel on interrupt. This allows multiple goroutines to wait on the event by waiting for the channel to close.
shutdown := make(chan struct{})
interrupt := make(chan os.Signal, 1)
signal.Notify(interrupt, os.Interrupt)
go func() {
<-interrupt
log.Println("interrupt")
close(shutdown)
}()
Move the per-connection code to a function. This code is a copy and paste from the question with two changes: the interrupt channel is replaced with the shutdown channel; the function notifies a sync.WaitGroup when the function is done.
func connect(u string, shutdown chan struct{}, wg *sync.WaitGroup) {
defer wg.Done()
log.Printf("connecting to %s", u)
c, _, err := websocket.DefaultDialer.Dial(u, nil)
if err != nil {
log.Fatal("dial:", err)
}
defer c.Close()
done := make(chan struct{})
go func() {
defer close(done)
for {
_, message, err := c.ReadMessage()
if err != nil {
log.Println("read:", err)
return
}
log.Printf("recv: %s", message)
}
}()
ticker := time.NewTicker(time.Second)
defer ticker.Stop()
for {
select {
case <-done:
return
case t := <-ticker.C:
err := c.WriteMessage(websocket.TextMessage, []byte(t.String()))
if err != nil {
log.Println("write:", err)
return
}
case <-shutdown:
// Cleanly close the connection by sending a close message and then
// waiting (with timeout) for the server to close the connection.
err := c.WriteMessage(websocket.CloseMessage, websocket.FormatCloseMessage(websocket.CloseNormalClosure, ""))
if err != nil {
log.Println("write close:", err)
return
}
select {
case <-done:
case <-time.After(time.Second):
}
return
}
}
}
Declare a sync.WaitGroup in main(). For each websocket endpoint that you want to connect to, increment the WaitGroup and start a goroutine to connect that endpoint. After starting the goroutines, wait on the WaitGroup for the goroutines to complete.
var wg sync.WaitGroup
for _, u := range endpoints { // endpoints is []string
// where elements are URLs
// of endpoints to connect to.
wg.Add(1)
go connect(u, shutdown, &wg)
}
wg.Wait()
The code above with an edit to make it run against Gorilla's echo example server is posted on the playground.
is the communication with every different server completely independendant of the other servers? if yes i would go around in a fashion like:
in main create a context with a cancellation function
create a waitgroup in main to track fired up goroutines
for every server, add to the waitgroup, fire up a new goroutine from the main function passing the context and the waitgroup references
main goes in a for/select loop listening to for signals and if one arrives calls the cancelfunc and waits on the waitgroup.
main can also listen on a result chan from the goroutines and maybe print the results itself it the goroutines shouldn't do it directly.
every goroutine has as we said has references for the wg, the context and possibly a chan to return results. now the approach splits on if the goroutine must do one and one thing only, or if it needs to do a sequence of things. for the first approach
if only one thing is to be done we follow an approach like the one descripbed here (observe that to be asyncronous he would in turn fire up a new goroutine to perform the DoSomething() step that would return the result on the channel)
That allows it to be able to accept the cancellation signal at any time. it is up to you to determine how non-blocking you want to be and how prompt you want to be to respond to cancellation signals.Also the benefit of having the a context associated being passed to the goroutines is that you can call the Context enabled versions of most library functions. For example if you want your dials to have a timeout of let's say 1 minute, you would create a new context with timeout from the one passed and then DialContext with that. This allows the dial to stop both from a timeout or the parent (the one you created in main) context's cancelfunc being called.
if more things need to be done ,i usually prefer to do one thing with the goroutine, have it invoke a new one with the next step to be performed (passing all the references down the pipeline) and exit.
this approach scales well with cancellations and being able to stop the pipeline at any step as well as support contexts with dealines easily for steps that can take too long.
I'm having the following server code, which listens via unix domain socket
package main
import (
"log"
"net"
"os"
"os/signal"
"syscall"
)
func echoServer(c net.Conn) {
for {
buf := make([]byte, 512)
nr, err := c.Read(buf)
if err != nil {
return
}
data := buf[0:nr]
println("Server got:", string(data))
_, err = c.Write(data)
if err != nil {
log.Fatal("Writing client error: ", err)
}
}
}
func main() {
log.Println("Starting echo server")
ln, err := net.Listen("unix", "/tmp/go.sock")
if err != nil {
log.Fatal("Listen error: ", err)
}
sigc := make(chan os.Signal, 1)
signal.Notify(sigc, os.Interrupt, syscall.SIGTERM)
go func(ln net.Listener, c chan os.Signal) {
sig := <-c
log.Printf("Caught signal %s: shutting down.", sig)
ln.Close()
os.Exit(0)
}(ln, sigc)
for {
fd, err := ln.Accept()
if err != nil {
log.Fatal("Accept error: ", err)
}
go echoServer(fd)
}
}
When I close it using Ctrl+C, then the signal is captured and the socket is closed. When I rerun the program, everything works fine.
However, if the running process is abruptly killed, and if the program is restarted, the listen fails with the error Listen error: listen unix /tmp/go.sock: bind: address already in use
How to graciously handle this?
The reason why I ask this is: I know that abruptly killing is not the normal method, but my program shall be launched automatically as a daemon and if my daemon is restarted, I want to be able to listen to the socket again without this error.
It could also be because of a prior instance running, which I understand. The question here is how to programmatically identify in Go and handle this situation. As pointed in the answer here, one can use SO_REUSEADDR in C programs. Is there such a possibility in Go? Also, how do C programs handle this multiple instance problem.
You need to catch the signal and cleanup; some example code:
func HandleSIGINTKILL() chan os.Signal {
sig := make(chan os.Signal, 1)
signal.Notify(sig, syscall.SIGINT, syscall.SIGTERM)
return sig
}
...
go func() {
<-HandleSIGINTKILL()
log.Info("Received termination signal")
// Cleanup code here
os.Exit(0)
}()
This will of course not work if you kill -9 the process; you will need to manually remove the socket (or have your init system do it for you).