Deploy a Go gRPC/REST Microservice to Kubernetes (GKE)
Recently I have been experimenting with Go applications and deployment to Kubernetes. While there are a lot of articles on both these subjects, I came across a three part article on how to create a Go microservice, which outlined a comprehensive approach to creating such a service and details significantly more than just a simple HelloWorld application.
The author of the three part article had intended to publish part 4, outlining how to deploy this service to Kubernetes. However, it seems as though part 4 has not been completed for some time, and there are many readers asking for this content. I decided to take it upon myself to see if I could create such a service on Google Kubernetes Engine (GKE).
This article begins where Part 3 of the above series left off. It assumes that one has an SQL database deployed and it is accessible via <IPaddr>:Port. In my case, the SQL DB instance is running in GCP.
The following outlines the step-by-step process I used. I confirmed each step’s efficacy locally on my Docker desktop before deployment to GKE. It also includes the Dockerfile used to create the Microservice container and the YAML templates used for the Kubernetes cluster.
There are five main steps to creating a cloud deployment of such a service:
- Creation of a Docker container and testing on Docker Desktop
- Hosting of the docker container within DockerHub
- Creation of the YAML deployment file and service files.
- Usage of the YAML template files to deploy the Microservice in Docker Desktop cluster
- Deployment of the todo Microservice to GKE
Step 1: Creation of a Docker container and testing on Docker Desktop
I use Mac for my development on which I have installed Docker desktop for running and testing docker containers. If you use other platforms, you will need to adapt the steps outlined here to your setup.
When creating the docker container, it is important to remove any unnecessary code not required in the container to make it as small as possible. The following article outlines the benefits and how this can be achieved using a .dockerignore file.
Create the .dockerignore file at the top level of your go-grpc-http-rest-microservice-tutorial directory with the following content.
# ignore .git and .cache folders
.git.cache# ignore client code./cmd/client-rest./cmd/client-grpc.Dockerfile# ignore Visual code workspace file
.go-grps-http-rest-microservice.code-workspace# ignore kubernetes deployment files
.todo-deployment.yaml.todo-service.yaml
Create the Docker file at the top level of your go-grpc-http-rest-microservice-tutorial directory with the following content. The content should be saved in file named Dockerfile.
# get the base image from docker hub -
# https://hub.docker.com/_/golangFROM golang# copy into the image the sourceCOPY . /go/src/go-grpc-http-rest-microservice-tutorial# set working dir for the main server functionWORKDIR /go/src/go-grpc-http-rest-microservice-tutorial/cmd/server# fetch the dependencies based on the main.goRUN go get .# run the main go function passing in all the necessary parametersENTRYPOINT ["go", "run", "main.go", "-grpc-port=9090", "-http-port=8080", "-db-host=<IPaddr>:3306", "-db-user=<username>", "-db-password=<mypass>", "-db-schema=ToDoTutorial", "-log-level=-1", "-log-time-format=2006-01-02T15:04:05.999999999Z07:00"]# expose the container ports to the outside world# note that on mac the container bridge does not work
# and port forwarding will be required as per
# https://docs.docker.com/docker-for-mac/networking/#known-limitations-use-cases-and-workaroundsEXPOSE 8080EXPOSE 9090
Be sure to replace the <IPAddr> with your own SQL DB IP. Similarly <username> and <password> for your own SQL DB.
Build the docker container with the following command. Replace <your_dockerid> with your docker-hub ID.
docker build . -t <your_dockerid>/go-grpc-http-rest-microservice-tutorialThe output should look something similar to what is shown below:
Sending build context to Docker daemon 16.5MBStep 1/7 : FROM golang---> 6d8772fbd285Step 2/7 : COPY . /go/src/go-grpc-http-rest-microservice-tutorial---> 23feab1dab37Step 3/7 : WORKDIR /go/src/go-grpc-http-rest-microservice-tutorial/cmd/server---> Running in 52879e89ab6fRemoving intermediate container 52879e89ab6f---> 29772667d04aStep 4/7 : RUN go get .---> Running in e0462a9c205dgo: downloading github.com/grpc-ecosystem/grpc-gateway v1.16.0go: downloading go.uber.org/zap v1.16.0go: downloading github.com/golang/protobuf v1.4.3go: downloading google.golang.org/grpc v1.33.2go: downloading go.uber.org/atomic v1.6.0go: downloading go.uber.org/multierr v1.5.0go: downloading github.com/grpc-ecosystem/go-grpc-middleware v1.2.2go: downloading github.com/go-sql-driver/mysql v1.5.0go: downloading google.golang.org/genproto v0.0.0-20201119123407-9b1e624d6bc4go: downloading google.golang.org/protobuf v1.25.0go: downloading golang.org/x/net v0.0.0-20200822124328-c89045814202go: downloading golang.org/x/sys v0.0.0-20200420163511-1957bb5e6d1fgo: downloading golang.org/x/text v0.3.2Removing intermediate container e0462a9c205d---> d371f6cb8b83Step 5/7 : ENTRYPOINT ["go", "run", "main.go", "-grpc-port=9090", "-http-port=8080", "-db-host=<...>:3306", "-db-user=<...>", "-db-password=<...>", "-db-schema=ToDoTutorial", "-log-level=-1", "-log-time-format=2006-01-02T15:04:05.999999999Z07:00"]---> Running in 073c8cce9260Removing intermediate container 073c8cce9260---> 2305c4881eedStep 6/7 : EXPOSE 8080---> Running in a77f6107a100Removing intermediate container a77f6107a100---> e4118ba4a4ddStep 7/7 : EXPOSE 9090---> Running in c06e7f7226e8Removing intermediate container c06e7f7226e8---> 3d2f30edb033Successfully built 3d2f30edb033
To test the container we just built, go to your Docker Desktop dashboard, images tab and click on the RUN button on right of the image: <yourname>/go-grpc-http-rest-microservice-tutorial:latest
Note: On Mac, port forwarding is required. Select Optional Setting after the Run and map Container Port 8080 to Local Host port 80 and Container Port 9090 to Local Host port 90. Then click Run.
Once the container is running, click on the running image to see the logs being output by the container. Client API calls will also be shown in the same log window.
{"level":"info","ts":"2020-12-14T23:00:40.36755111Z","msg":"[core]parsed scheme: \"\"","system":"grpc","grpc_log":true}
{"level":"info","ts":"2020-12-14T23:00:40.367907561Z","msg":"[core]scheme \"\" not registered, fallback to default scheme","system":"grpc","grpc_log":true}
{"level":"info","ts":"2020-12-14T23:00:40.367791819Z","msg":"starting gRPC server..."}
{"level":"info","ts":"2020-12-14T23:00:40.370524486Z","msg":"[core]ccResolverWrapper: sending update to cc: {[{localhost:9090 <nil> 0 <nil>}] <nil> <nil>}","system":"grpc","grpc_log":true}
{"level":"info","ts":"2020-12-14T23:00:40.370647291Z","msg":"[core]ClientConn switching balancer to \"pick_first\"","system":"grpc","grpc_log":true}
{"level":"info","ts":"2020-12-14T23:00:40.370661232Z","msg":"[core]Channel switches to new LB policy \"pick_first\"","system":"grpc","grpc_log":true}
{"level":"info","ts":"2020-12-14T23:00:40.372479399Z","msg":"[core]Subchannel Connectivity change to CONNECTING","system":"grpc","grpc_log":true}
{"level":"info","ts":"2020-12-14T23:00:40.372727844Z","msg":"[core]pickfirstBalancer: UpdateSubConnState: 0xc00003c1f0, {CONNECTING <nil>}","system":"grpc","grpc_log":true}
{"level":"info","ts":"2020-12-14T23:00:40.372793802Z","msg":"[core]Channel Connectivity change to CONNECTING","system":"grpc","grpc_log":true}
{"level":"info","ts":"2020-12-14T23:00:40.373003054Z","msg":"[core]Subchannel picks a new address \"localhost:9090\" to connect","system":"grpc","grpc_log":true}
{"level":"info","ts":"2020-12-14T23:00:40.374992001Z","msg":"starting HTTP/REST gateway..."}
{"level":"info","ts":"2020-12-14T23:00:40.376456582Z","msg":"[core]Subchannel Connectivity change to READY","system":"grpc","grpc_log":true}
{"level":"info","ts":"2020-12-14T23:00:40.376756458Z","msg":"[core]pickfirstBalancer: UpdateSubConnState: 0xc00003c1f0, {READY <nil>}","system":"grpc","grpc_log":true}
{"level":"info","ts":"2020-12-14T23:00:40.376986392Z","msg":"[core]Channel Connectivity change to READY","system":"grpc","grpc_log":true}
...Go to the respective rest and grpc client directories and run the following commands to verify the CRUD API calls are successful.
./client-rest -server=http://localhost:80./client-grpc -server=localhost:90
Step 2: Hosting of the docker container within DockerHub
Before running the command below, you will need to log into your Docker Hub account from Docker Desktop.
docker push <your_dockerid>/go-grpc-http-rest-microservice-tutorialGo to your Docker Hub portal using a browser and verify that the image is uploaded correctly.
Step 3: Creation of the YAML deployment file and service file
Now that we have verified that the Docker container works, the container must be stopped so that we can deploy a Kubernetes cluster running the same container and use those same exposed ports.
Go to the Docker Desktop dashboard, hover over the image, and click the STOP icon.
Create the Kubernetes deployment file at the top level of your go-grpc-http-rest-microservice-tutorial directory with the following content. The following content should be saved in file named todo-deployment.yaml. Also, make sure to update the <your-dockerid> in the file to point to your docker hub repo.
Create the Kubernetes service file at the top level of your go-grpc-http-rest-microservice-tutorial directory with the following content.
This file defines the service which will route the API calls to the todo-container.
Step 4: Usage of the YAML template files to deploy the Microservice in Docker Desktop cluster
Run the following commands from your terminal window to create the Kubernetes pod with the service running:
kubectl apply -f todo-deployment.yamlkubectl apply -f todo-service.yaml
Once the todo pod is in a running state, you can view the logs from the gRPC/REST microservice by clicking the k8s_todo_todo-…, which will take you to the window shown below.
Go to the respective rest and grpc client directories and run the commands below to verify the CRUD API calls are successful.
./client-grpc -server=localhost:30090./client-rest -server=http://localhost:30080
Step 5: Deployment of the todo Microservice to GKE
Before the todo service can be deployed, a few infrastructure setup steps need to be taken:
a) Ensure you have installed Google cloud SDK as per the steps outlined in the Google docs and that you have a project setup
b) Create a cluster in Google Cloud so that the service can be deployed in that cluster.
gcloud container clusters create todo-microservice-clusterc) The service in Google Cloud will be running in clusters whose IP addresses must be authorized in SQL database configuration so that the Microservice can access the SQL DB.
Go to the Google console, Compute Engine-> VM Instances to see the clusters. Click on each node within the cluster to get the External IP (ephemeral).
Go to SQL->Connections and add each Public IP to authorize access from an external IP
cluster-node1: <IPAddr1>/32
cluster-node2: <IPAdde2>/32
cluster-node3: <IPAddr3>/32
d) Add a Firewall rule to allow access to the gRPC and REST microservices ports: 30080, 30090 respectively. Go to VPC Network->Firewall. Create Firewall rule named todo-microservice, Targets: all instances in the network, source IP should be set to the IP address of the client machine and enable TCP and add the following ports: 30090, 30080.
Once the above steps are completed, the infrastructure is ready to deploy the todo Microservice. Use the following commands to deploy:
kubectl apply -f todo-deployment.yaml kubectl apply -f todo-service.yaml
Go to the respective rest and grpc client directories and run the following commands to verify the CRUD API calls are successful. Note the <cluster-node-public-ip> must be replaced with any one of the IP addresses that were obtained from step 5(c) above.
./client-rest -server=http://<cluster-node-public-ip>:30080./client-grpc -server=<cluster-node-public-ip>:30090
Debugging Potential Issues
If the client APIs fail in your setup, it will require you to debug the problem. Debugging GKE is not straightforward, but here are few pointers which will help in the debug process.
a) Check that the service is running by running the following command. It should show the NAME, TYPE, CLUSTER-IP, EXTERNAL-IP, PORT(s). Make sure the ports match 8080:30080/TCP,9090:30090/TCP
kubectl get service todo-serviceb) Connect to the todo container and confirm that you are able to ping the DB. This will confirm if the connectivity from the pod to the database instance is working.
kubectl get pods# the above will list the container name
# e.g. todo-container-6d8bd89b79-w82zckubectl exec -it <container-name> -c todo /bin/sh#once connected to the container shell
ping <public IP DB>exit
c) Look at the logs by going to GCP console
Logging->Log Explorer
Create a query by selecting Resource button->pod->us-west1->your-cluster
Resource button->container-<your-cluster>
This will allow you to view logs output from the container and any API calls.
Production deployment
This article described how to deploy the todo microservice to GKE for testing purposes, but it is not intended to support a production environment. There are a couple of deficiencies: a) The service is using NodePort, which means that all requests will be directed to the same node and if the node goes down then the client will need to use an IP address of one of the other nodes; b) The service, as currently setup, is not secure. Using ingress service, which supports secure connections using Global Load Balancer (GLB), it would be possible to secure both HTTP and gRPC connections.
