Docker: go get from a private GitHub repo - github

Question

I'm trying to run a container that will expose a golang service from a package that I have on a private GitHub repo.
Since I am working with GCE, my starter image is google/debian:wheezy.
After installing all the required dependancies and tools, I am running
RUN go get github.com/<my_org>/<my_package>
where the package is a private repo.
I have added my GitHub SSH keys to allow the cloning from the private repo to the docker file:
ADD priv/id_rsa /root/.ssh/id_rsa
ADD priv/id_rsa.pub /root/.ssh/id_rsa.pub
Still, I am getting an error during the go get process when go tried to clone the repo:
# cd .; git clone https://github.com/<my_org>/<my_package> /gopath/src/github.com/<my_org>/<my_package>
Cloning into '/gopath/src/github.com/<my_org>/<my_package>'...
fatal: could not read Username for 'https://github.com': No such device or address
package github.com/<my_org>/<my_package>: exit status 128
To debug the problem, from the Dockerfile, I am running:
RUN ssh-keyscan -t rsa github.com 2>&1 >> /root/.ssh/known_hosts
And this tells me there are some problems. It looks like validating the private key is OK but something weird is going on the the public key. This is the complete ssh-keyscan result:
OpenSSH_6.0p1 Debian-4+deb7u2, OpenSSL 1.0.1e 11 Feb 2013
Pseudo-terminal will not be allocated because stdin is not a terminal.
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to github.com [192.30.252.129] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/id_rsa type 1
debug1: Checking blacklist file /usr/share/ssh/blacklist.RSA-2048
debug1: Checking blacklist file /etc/ssh/blacklist.RSA-2048
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: Remote protocol version 2.0, remote software version libssh-0.6.0
debug1: no match: libssh-0.6.0
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.0p1 Debian-4+deb7u2
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-sha1 none
debug1: kex: client->server aes128-ctr hmac-sha1 none
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: RSA 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48
debug1: Host 'github.com' is known and matches the RSA host key.
debug1: Found key in /root/.ssh/known_hosts:1
Warning: Permanently added the RSA host key for IP address '192.30.252.129' to the list of known hosts.
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /root/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 279
debug1: key_parse_private_pem: PEM_read_PrivateKey failed
debug1: read PEM private key done: type <unknown>
debug1: read_passphrase: can't open /dev/tty: No such device or address
debug1: Trying private key: /root/.ssh/id_dsa
debug1: Trying private key: /root/.ssh/id_ecdsa
debug1: No more authentication methods to try.
Permission denied (publickey).
I have tried chmod 600 and chmod 700 on the priv/public keys, this did not help.
Any clues? Has anyone succeeding in running go get that fetches from private repos on debian from docker?

Answer 1

I figured this out after a bit of hacking around. Not an ideal solution as it involves installing SSH, plus building a private key into the container. This example is based on the official Docker golang image (Debian Wheezy):
The main difference to your example is that you need a git config command to force ssh instead of the default https.
FROM golang
RUN apt-get update && apt-get install -y ca-certificates git-core ssh
ADD keys/my_key_rsa /root/.ssh/id_rsa
RUN chmod 700 /root/.ssh/id_rsa
RUN echo "Host github.com\n\tStrictHostKeyChecking no\n" >> /root/.ssh/config
RUN git config --global url.ssh://git#github.com/.insteadOf https://github.com/
ADD . /go/src/github.com/myaccount/myprivaterepo
RUN go get github.com/myaccount/myprivaterepo
RUN go install github.com/myaccount/myprivaterepo

Answer 2

go get is trying to use https, completely ignoring ssh.
You will have to setup ~/.netrc:
ADD priv/.netrc /root/.netrc
Where netrc looks like:
machine github.com login github-username password github-password
ref:
https://stackoverflow.com/a/13724351/145587

Answer 3

In the newest version of golang (v1.11) there are now modules.
To quote the source:
A module is a collection of related Go packages that are versioned together as a single unit. Most often, a single version-control repository corresponds exactly to a single module.
Using the latest version of golang will allow you to have dependencies that are in private repositories. Essentially by running the $ go mod vendor command will create a vendor directory locally for all external dependencies. Now making sure your docker image has Golang v1.11, you will update your Dockerfile with the following:
WORKDIR /<your repostiory>
COPY . ./

Answer 4

Elaborating on OneOfOne's ~/.netrc answer, this is what I am doing with Jenkins on linux:
FROM golang:1.6
ARG GITHUB_USER=$GITHUB_USER
ARG GITHUB_PASS=$GITHUB_PASS
# Copy local package files to the container's workspace.
ADD . /go/src/github.com/my-org/my-project
WORKDIR /go/src/github.com/my-org/my-project/
# Build application inside the container.
RUN echo "machine github.com\n\tlogin $GITHUB_USER\n\tpassword $GITHUB_PASS" >> ~/.netrc && \
go get github.com/tools/godep && \
go get github.com/onsi/ginkgo/ginkgo && \
godep restore && \
ginkgo -r --randomizeAllSpecs --randomizeSuites --failOnPending && \
godep go install && \
rm -f ~/.netrc
ENTRYPOINT /go/bin/my-project
EXPOSE 8080
The docker build command is:
docker build \
--build-arg GITHUB_USER=xxxxx \
--build-arg GITHUB_PASS=yyyyy \
-t my-project .
The two ARG directives map --build-args so docker can use them inside the Dockerfile.
The first and last lines of RUN create and remove the ~/.netrc.
In Jenkins, I use the same creds from git pull in the build command.
In this strategy, the password is not echoed during the docker build process and not saved on any layer of your docker image. Also note that the gingko test results are printed to console during the build.

Answer 5

i had this problem in Github and i fix it using personal access token:
first of all please use ARG for your Dockerfile vars(inputs):
after that configure your git with github personal access token
GITHUB_PAT -> github personal access token
FROM golang:1.17 as builder
ARG GITHUB_PAT
WORKDIR /your-app
COPY go.mod .
COPY go.sum .
RUN git config --global url."https://${GITHUB_PAT}:x-oauth-basic#github.com/".insteadOf "https://github.com/"
RUN go mod download
COPY . .
RUN go build -ldflags '-w -s' -o ./out ./main.go
FROM golang:1.17
WORKDIR /app
COPY --from=builder /your-app/out ./
WORKDIR /app/
ENTRYPOINT [ "./out" ]