基于golang从头开始构建基于docker的微服务实战笔记

概述参考博文 part 1 利用gRPC protobuf定义服务 part 2 - Docker and go-micro Go-micro part 3 - docker-compose and datastores Part 4 - Authentication with JWT JWT User-service consignment-cli consignment-server Part 5 参考博文 part 1 利用gRPC protobuf定义服务 part 2 - Docker and go-micro Go-micro part 3 - docker-compose and datastores Part 4 - Authentication with JWT JWT User-service consignment-cli consignment-server Part 5 - Event brokering with Go Micro NATS配置 user-service nats连接失败 参考博文

https://ewanvalentine.io/microservices-in-golang-part-1/
这个博文是作者微服务系统的第一篇，本学习笔记基于目前的5篇而成

part 1 利用gRPC,protobuf定义服务

本人在学习过程中没有严格按照博文中放在github目录，而是在主目录中创建一个wdyshippy的目录，目录中文件结构如下

.├── consignment-cli│   ├── cli.go│   ├── consignment-cli│   ├── consignment.Json│   ├── Dockerfile│   └── Makefile└── consignment-service    ├── consignment-service    ├── Dockerfile    ├── main.go    ├── Makefile    └── proto        └── consignment            ├── consignment.pb.go            └── consignment.proto4 directorIEs,11 files

执行的protoc命令

protoc -I. --go_out=plugins=grpc:/home/wdy/wdyshippy/consignment-service proto/consignment/consignment.proto

如果make build时报错

makefile:2: *** 遗漏分隔符

原因是在编写makefile文件时，命令前面应该是tab写为了空格

在这个阶段中，consignment-service的Makefile文件内容如下

build:    protoc -I. --go_out=plugins=grpc:$(GOPATH)/src/github.com/ewanvalentine/shipper/consignment-service \      proto/consignment/consignment.proto

part 2 - Docker and go-micro

创建DockefIEl

$ touch consignment-service/Dockerfile

写入如下内容

FROM alpine:latestRUN mkdir /appworkdir /appADD consignment-service /app/consignment-serviceCMD ["./consignment-service"]

原作者提示如果是在linux机子上编译测试时，将alpine改为debian

FROM debian:latestRUN mkdir /appworkdir /appADD consignment-service /app/consignment-serviceCMD ["./consignment-service"]

workdir 表示 ／app目录作为上下文目录用来装载我们的程序consignment-service

接下来为Makefile文件增加内容用来生成docker image

build:    ...     GOOS=linux GOARCH=amd64 go build    docker build -t consignment-service .

除此之外再添加

run:     docker run -p 50051:50051 consignment-service

用来执行consignment-service

docker run -p 后面的参数，第一个端口号是对外的端口，第二个是内部的端口。

Go-micro

使用Go-micro来加入service discovery功能

首先安装

go get -u github.com/micro/protobuf/{proto,protoc-gen-go}

修改makefile文件中protoc的参数,将参数plugins=后面的grpc更改为micro

build:    protoc -I. --go_out=plugins=micro:/home/wdy/wdyshippy/consignment-service proto/consignment/consignment.proto    ......

代码也要相应修改

import中 要引入 micro “github.com/micro/go-micro”

// consignment-service/main.gopackage mainimport (    // import the generated protobuf code    "fmt"    pb "github.com/EwanValentine/shippy/consignment-service/proto/consignment"    micro "github.com/micro/go-micro"    "golang.org/x/net/context")

server接口实现时，response的位置改变为输入参数，输出参数只有error

func (s *service) CreateConsignment(ctx context.Context,req *pb.Consignment,res *pb.Response) error {...func (s *service) GetConsignments(ctx context.Context,req *pb.GetRequest,res *pb.Response) error {  ...

还有就是server的初始化

func main() {    repo := &Repository{}    // Create a new service. Optionally include some options here.    srv := micro.NewService(        // This name must match the package name given in your protobuf deFinition        micro.name("go.micro.srv.consignment"),micro.Version("latest"),)    // Init will parse the command line flags.    srv.Init()    // Register handler    pb.RegisterShipPingServiceHandler(srv.Server(),&service{repo})    // Run the server    if err := srv.Run(); err != nil {        fmt.Println(err)    }}

最后就是不需要在代码中硬编码端口号，Go-micro通过环境变量或命令行参数来传递。

run:    docker run -p 50051:50051 \        -e MICRO_SERVER_ADDRESS=:50051 \        -e MICRO_REGISTRY=mdns consignment-service

如上，通过 -e MICRO_SERVER_ADDRESS=:50051 指定服务地址，通过-e MICRO_REGISTRY=mdns指定service discovery功能使用mdns。在实际项目中mdns很少使用，大部分使用consul。

除了服务端代码更改外，客户端代码也需要更改

import (    ...    "github.com/micro/go-micro/cmd"    microclIEnt "github.com/micro/go-micro/clIEnt")func main() {    cmd.Init()    // Create new greeter clIEnt    clIEnt := pb.NewShipPingServiceClIEnt("go.micro.srv.consignment",microclIEnt.DefaultClIEnt)    ...}

通过docker启动consignment-service后，执行客户端程序会连接失败，原因在于server允许在docker中使用的是docker中的dmns，和客户端使用的不是同一个，所以发现不了server，解决方法就是把客户端程序也放入到docker 中，使用同一个dmns。

创建consignment-cli/Makefile文件

build:    GOOS=linux GOARCH=amd64 go build    docker build -t consignment-cli .run:    docker run -e MICRO_REGISTRY=mdns consignment-cli

Dockerfile文件

FROM alpine:latestRUN mkdir -p /appworkdir /appADD consignment.Json /app/consignment.JsonADD consignment-cli /app/consignment-cliCMD ["./consignment-cli"]

作者最后又提供了一个更加标准的Dockerfile文件，这个文件中包含了consignment-service的开发环境和生存环境

在开发环境中通过引用github.com/golang/dep/cmd/dep 来自动更新包依赖

RUN go get -u github.com/golang/dep/cmd/dep# Create a dep project,and run `ensure`,which will pull in all # of the dependencIEs within this directory.RUN dep init && dep ensure

完整文件如下

# consignment-service/Dockerfile# We use the official golang image,which contains all the # correct build tools and librarIEs. Notice `as builder`,# this gives this container a name that we can reference later on. FROM golang:1.9.0 as builder# Set our workdir to our current service in the gopathworkdir /go/src/github.com/EwanValentine/shippy/consignment-service# copy the current code into our workdircopY . .# Here we're pulling in godep,which is a dependency manager tool,# we're going to use dep instead of go get,to get around a few# quirks in how go get works with sub-packages.RUN go get -u github.com/golang/dep/cmd/dep# Create a dep project,and run `ensure`,which will pull in all # of the dependencIEs within this directory.RUN dep init && dep ensure# Build the binary,with a few flags which will allow# us to run this binary in Alpine. RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo .# Here we're using a second FROM statement,which is strange,# but this tells Docker to start a new build process with this# image.FROM alpine:latest# Security related package,good to have.RUN apk --no-cache add ca-certificates# Same as before,create a directory for our app.RUN mkdir /appworkdir /app# Here,instead of copying the binary from our host machine,# we pull the binary from the container named `builder`,within# this build context. This reaches into our prevIoUs image,finds# the binary we built,and pulls it into this container. Amazing!copY --from=builder /go/src/github.com/EwanValentine/shippy/consignment-service/consignment-service .# Run the binary as per usual! This time with a binary build in a# separate container,with all of the correct dependencIEs and# run time librarIEs.CMD ["./consignment-service"]

part 3 - docker-compose and datastores

介绍了docker-compose的安装 Install docker-compose: https://docs.docker.com/compose/install/

还有 docker-compose的使用

docker-compose.yml的内容如下：

version: '3.1'services: consignment-cli: build: ./consignment-cli environment: MICRO_REGISTRY: "mdns" consignment-service: build: ./consignment-service ports: - 50051:50051 environment: MICRO_ADDRESS: ":50051" MICRO_REGISTRY: "mdns" DB_HOST: "datastore:27017" vessel-service: build: ./vessel-service ports: - 50052:50051 environment: MICRO_ADDRESS: ":50051" MICRO_REGISTRY: "mdns"

还介绍了数据库包

http://jinzhu.me/gorm/

最后介绍了go-micro客户端另外一种编写方式

package mainimport (    "log"    "os"    pb "github.com/EwanValentine/shippy/user-service/proto/user"    microclIEnt "github.com/micro/go-micro/clIEnt"    "github.com/micro/go-micro/cmd"    "golang.org/x/net/context"    "github.com/micro/cli"    "github.com/micro/go-micro")func main() {    cmd.Init()    // Create new greeter clIEnt    clIEnt := pb.NewUserServiceClIEnt("go.micro.srv.user",microclIEnt.DefaultClIEnt)    // define our flags    service := micro.NewService(        micro.Flags(            cli.StringFlag{                name:  "name",Usage: "You full name",},cli.StringFlag{                name:  "email",Usage: "Your email",cli.StringFlag{                name:  "password",Usage: "Your password",cli.StringFlag{                name: "company",Usage: "Your company",),)    // Start as service    service.Init(        micro.Action(func(c *cli.Context) {            name := c.String("name")            email := c.String("email")            password := c.String("password")            company := c.String("company")            // Call our user service            r,err := clIEnt.Create(context.Todo(),&pb.User{                name: name,Email: email,Password: password,Company: company,})            if err != nil {                log.Fatalf("Could not create: %v",err)            }            log.Printf("Created: %s",r.User.ID)            getAll,err := clIEnt.GetAll(context.Background(),&pb.Request{})            if err != nil {                log.Fatalf("Could not List users: %v",err)            }            for _,v := range getAll.Users {                log.Println(v)            }            os.Exit(0)        }),)    // Run the server    if err := service.Run(); err != nil {        log.Println(err)    }}

传递参数运行如下

$ docker-compose run user-cli command \  --name="Ewan Valentine" \   --email="[email protected]" \   --password="Testing123" \   --company="BBC"

Part 4 - Authentication with JWT

运行两个数据库

$ docker run -d -p 5432:5432 postgres$ docker run -d -p 27017:27017 mongo

JWT

https://jwt.io/

http://cryto.net/~joepie91/blog/2016/06/13/stop-using-jwt-for-sessions/

Cheat_Sheet_for_Java”>https://www.owasp.org/index.php/JSON_Web_Token(JWT)_Cheat_Sheet_for_Java

https://auth0.com/blog/json-web-token-signing-algorithms-overview/

https://tools.ietf.org/html/rfc7518#section-3

Go library for this: github.com/dgrijalva/jwt-go

User-service

负责用户信息的认证和token的发放校验

https://github.com/EwanValentine/shippy/blob/master/user-service/token_service.go

Token_service.go中代码负责jwt的编码(Encode)和解码(Decode)，用于server的调用

Encode

// Encode a claim into a JWTfunc (srv *TokenService) Encode(user *pb.User) (string,error) {    // Create the Claims    claims := CustomClaims{        user,jwt.StandardClaims{            ExpiresAt: 15000,Issuer:    "go.micro.srv.user",}    // Create token    token := jwt.NewWithClaims(jwt.SigningMethodHS256,claims)    // Sign token and return    return token.SignedString(key)}

上面代码有误，设置ExpiresAt 为15000，运行程序会报token过期，正确代码为

// Encode a claim into a JWTfunc (srv *TokenService) Encode(user *User) (string,error) {    expiretoken := time.Now().Add(time.Hour * 72).Unix()    // Create the Claims    claims := CustomClaims{        user,jwt.StandardClaims{            ExpiresAt: expiretoken,claims)    // Sign token and return    return token.SignedString(key)}

Decode

func (srv *TokenService) Decode(token string) (*CustomClaims,error) {    // Parse the token    tokenType,err := jwt.ParseWithClaims(string(key),&CustomClaims{},func(token *jwt.Token) (interface{},error) {        return key,nil    })    // ValIDate the token and return the custom claims    if claims,ok := tokenType.Claims.(*CustomClaims); ok && tokenType.ValID {        return claims,nil    } else {        return nil,err    }}

上面代码运行会panic

2018/01/17 01:39:31 panic recovered: runtime error: invalID memory address or nil pointer dereference2018/01/17 01:39:31 goroutine 35 [running]:runtime/deBUG.Stack(0xc420121920,0x2,0x2)        /home/wdy/go/src/runtime/deBUG/stack.go:24 +0x79github.com/micro/go-micro/server.(*rpcServer).accept.func1(0xb8dfe0,0xc4200e4f80)        /home/wdy/svn/cloud_trunk/factory/branches/tob_material/main/tobmaterialsys/src/github.com/micro/go-micro/server/rpc_server.go:60 +0x124panic(0x8d1720,0xbbe270)        /home/wdy/go/src/runtime/panic.go:489 +0x2cfmain.(*TokenService).Decode(0xc420011560,0xc4202cf080,0x15b,0xc420266af0,0xc42011e780,0xc4200326b8)        /home/wdy/wdyshippy/user-service/token_service.go:45 +0x10cmain.(*service).ValIDatetoken(0xc4200b1740,0x7f90d890c050,0xc420271140,0xc420271170,0xc420032738,0x41168c)        /home/wdy/wdyshippy/user-service/handler.go:76 +0x11c_/home/wdy/wdyshippy/user-service/proto/user.(*UserService).ValIDatetoken(0xc420011af0,0x0,0x0)        /home/wdy/wdyshippy/user-service/proto/user/user.pb.go:310 +0x5breflect.Value.call(0xc42005ea00,0xc42000e1d0,0x13,0x95de61,0x4,0xc420032bb0,0x901c80,0x913a40,...)        /home/wdy/go/src/reflect/value.go:434 +0x91freflect.Value.Call(0xc42005ea00,0x40,0x38)        /home/wdy/go/src/reflect/value.go:302 +0xa4github.com/micro/go-micro/server.(*service).call.func1(0x7f90d890c050,0xb8fdc0,0xc420132f00,0x0)

原因在于Decode中jwt.ParseWithClaims的第一个参数应该是token，正确代码为：

// Decode a token string into a token objectfunc (srv *TokenService) Decode(tokenString string) (*CustomClaims,error) {    // Parse the token    token,err := jwt.ParseWithClaims(tokenString,func(token *jwt.Token) (interface{},error) {        return key,nil    })    // ValIDate the token and return the custom claims    if claims,ok := token.Claims.(*CustomClaims); ok && token.ValID {        return claims,nil    } else {        return nil,err    }}

https://github.com/EwanValentine/shippy/blob/master/user-service/handler.go

负责用户的创建，密码校验和token发放和校验

golang.org/x/crypto/bcrypt

用户创建

func (srv *service) Create(ctx context.Context,req *pb.User,res *pb.Response) error {    // Generates a hashed version of our password    hashedPass,err := bcrypt.GenerateFromPassword([]byte(req.Password),bcrypt.DefaultCost)    if err != nil {        return err    }    req.Password = string(hashedPass)    if err := srv.repo.Create(req); err != nil {        return err    }    res.User = req    return nil}

用户密码校验，校验成功则发放token

func (srv *service) Auth(ctx context.Context,req *pb.User,res *pb.Token) error {    log.Println("Logging in with:",req.Email,req.Password)    user,err := srv.repo.GetByEmail(req.Email)    log.Println(user)    if err != nil {        return err    }    // Compares our given password against the hashed password    // stored in the database    if err := bcrypt.CompareHashAndPassword([]byte(user.Password),[]byte(req.Password)); err != nil {        return err    }    token,err := srv.tokenService.Encode(user)    if err != nil {        return err    }    res.Token = token    return nil}

token校验

func (srv *service) ValIDatetoken(ctx context.Context,req *pb.Token,res *pb.Token) error {    // Decode token    claims,err := srv.tokenService.Decode(req.Token)    if err != nil {        return err    }    log.Println(claims)    if claims.User.ID == "" {        return errors.New("invalID user")    }    res.ValID = true    return nil}

consignment-cli

客户端请求的时候加上token

github.com/micro/go-micro/Metadata...ctx := Metadata.NewContext(context.Background(),map[string]string{        "token": token,})    r,err := clIEnt.CreateConsignment(ctx,consignment)    if err != nil {        log.Fatalf("Could not create: %v",err)    }    log.Printf("Created: %t",r.Created)    getAll,err := clIEnt.GetConsignments(ctx,&pb.GetRequest{})    if err != nil {        log.Fatalf("Could not List consignments: %v",err)    }    for _,v := range getAll.Consignments {        log.Println(v)    }...

consignment-server

https://github.com/EwanValentine/shippy/tree/master/consignment-service

// shippy-consignment-service/main.gofunc main() {    ...     // Create a new service. Optionally include some options here.    srv := micro.NewService(        // This name must match the package name given in your protobuf deFinition        micro.name("go.micro.srv.consignment"),micro.Version("latest"),// Our auth mIDdleware        micro.WrapHandler(AuthWrapper),)    ...}... // AuthWrapper is a high-order function which takes a HandlerFunc// and returns a function,which takes a context,request and response interface.// The token is extracted from the context set in our consignment-cli,that// token is then sent over to the user service to be valIDated.// If valID,the call is passed along to the handler. If not,// an error is returned.func AuthWrapper(fn server.HandlerFunc) server.HandlerFunc {    return func(ctx context.Context,req server.Request,resp interface{}) error {        Meta,ok := Metadata.FromContext(ctx)        if !ok {            return errors.New("no auth Meta-data found in request")        }        // Note this is Now uppercase (not entirely sure why this is...)        token := Meta["Token"]        log.Println("Authenticating with token: ",token)        // Auth here        authClIEnt := userService.NewUserServiceClIEnt("go.micro.srv.user",clIEnt.DefaultClIEnt)        _,err := authClIEnt.ValIDatetoken(context.Background(),&userService.Token{            Token: token,})        if err != nil {            return err        }        err = fn(ctx,req,resp)        return err    }}

AuthWarpper是一个中间件，接受一个HandlerFunc，进行某种处理后 返回HandlerFunc。

该中间件从context提取token，然后发送到user-service中校验token是否有效，如果有效再继续执行真正的 *** 作fn。

Part 5 - Event brokering with Go Micro

修改 user-service/main.go

func main() {    ...     // Init will parse the command line flags.    srv.Init()    // Get instance of the broker using our defaults    pubsub := srv.Server().Options().broker    // Register handler    pb.RegisterUserServiceHandler(srv.Server(),&service{repo,tokenService,pubsub})    ...}

修改user-service/handler.go

const topic = "user.created"type service struct {    repo         Repository    tokenService Authable    PubSub       broker.broker}func (srv *service) Create(ctx context.Context,bcrypt.DefaultCost)    if err != nil {        return err    }    req.Password = string(hashedPass)    if err := srv.repo.Create(req); err != nil {        return err    }    res.User = req    if err := srv.publishEvent(req); err != nil {        return err    }    return nil}func (srv *service) publishEvent(user *pb.User) error {    // Marshal to JsON string    body,err := Json.Marshal(user)    if err != nil {        return err    }    // Create a broker message    msg := &broker.Message{        header: map[string]string{            "ID": user.ID,Body: body,}    // Publish message to broker    if err := srv.PubSub.Publish(topic,msg); err != nil {        log.Printf("[pub] Failed: %v",err)    }    return nil}

除此之外 user-service handler.go import添加

​```go"github.com/micro/go-micro/broker"_ "github.com/micro/go-plugins/broker/nats"​```

user-service/Makefile中添加

-e MICRO_broKER=nats \        -e MICRO_broKER_ADDRESS=0.0.0.0:4222 \

email-service参见原作者的repo https://github.com/EwanValentine/shippy-email-service

srv.Init()pubsub := srv.Server().Options().broker

在go-micro中， srv.Init()会搜索所有的配置，包括plugin，环境变量以及命令行参数，然后把这些配置初始化为service的组成部分。为了使用这些配置实例，需要通过 srv.Server().Options(),在本项目例子中，通过指定

-e MICRO_broKER=nats \        -e MICRO_broKER_ADDRESS=0.0.0.0:4222 \

go-micro会找到NATS的broker plugin，创建对应的实例，用于之后的连接和使用。

在email-server中，用来订阅event

if err := pubsub.Connect(); err != nil {        log.Fatal(err)    }    // Subscribe to messages on the broker    _,err := pubsub.Subscribe(topic,func(p broker.Publication) error {        var user *pb.User        if err := Json.Unmarshal(p.Message().Body,&user); err != nil {            return err        }        log.Println(user)        go sendEmail(user)        return nil    })

在user-service中，用来发布event

func (srv *service) publishEvent(user *pb.User) error {    // Marshal to JsON string    body,err)    }    return nil}

遇到的问题以及解决方法

NATS配置

本人成功的方法如下

wdy@wdy:~$ docker pull natsUsing default tag: latestlatest: Pulling from library/natsf169c9506d74: Pull completebb9eff5cafb0: Pull completeDigest: sha256:61fcb1f40da2111434fc910b0865c54155cd6e5f7c42e56e031c3f35a9998075Status: Downloaded newer image for nats:latestwdy@wdy:~$ docker run -p 4222:4222 -p 8222:8222 -p 6222:6222 --name gnatsd -ti nats:latest[1] 2018/01/17 05:45:07.167855 [INF] Starting nats-server version 1.0.4[1] 2018/01/17 05:45:07.167935 [INF] Starting http monitor on 0.0.0.0:8222[1] 2018/01/17 05:45:07.167961 [INF] Listening for clIEnt connections on 0.0.0.0:4222[1] 2018/01/17 05:45:07.167964 [INF] Server is ready[1] 2018/01/17 05:45:07.168111 [INF] Listening for route connections on 0.0.0.0:6222

user-service nats连接失败

User-service 执行make run之后报如下错误

nats: no servers available for connection

解决办法，将user_serice中的makefile的

run:    docker run -p 50053:50051 \    -e MICRO_SERVER_ADDRESS=:50051 \    -e MICRO_REGISTRY=mdns \    user-service

改为

run:    docker run --net="host" \    -p 50055 \    -e MICRO_SERVER_ADDRESS=:50055 \    -e MICRO_REGISTRY=mdns \    -e MICRO_broKER=nats \    -e MICRO_broKER_ADDRESS=0.0.0.0:4222 \

//Todo 等作者6-11的更新后，本文再同步更新

总结

以上是内存溢出为你收集整理的基于golang从头开始构建基于docker的微服务实战笔记全部内容，希望文章能够帮你解决基于golang从头开始构建基于docker的微服务实战笔记所遇到的程序开发问题。

如果觉得内存溢出网站内容还不错，欢迎将内存溢出网站推荐给程序员好友。