description
    自定义压缩

    绝大多数客户端都支持多种压缩方式,需要不同的场景选择适合的压缩算法,微服务之间的调用更是可以选择一些高性能的压缩方式,下面来介绍如何编写自定义的压缩中间件,主要的要点如下:

    • 根据请求头Accept-Encoding判断客户端支持的压缩算法
    • 设定最小压缩长度,避免对较小数据的压缩浪费性能
    • 根据响应头Content-Type判断只压缩文本类的响应数据
    • 根据场景平衡压缩率与性能的选择,如内网的可以选择snappy,lz4等高效压缩算法

    elton-compress中间件提供了其它几种常用的压缩方式,包括brotli以及snappy等。如果要增加压缩方式,只需要实现Compressor的三个函数则可。

    1. // Compressor compressor interface
    2. Compressor interface {
    3. // Accept accept check function
    4. Accept(c *elton.Context, bodySize int) (acceptable bool, encoding string)
    5. // Compress compress function
    6. Compress([]byte) (*bytes.Buffer, error)
    7. // Pipe pipe function
    8. Pipe(*elton.Context) error
    9. }

    下面是elton-compress中间件lz4压缩的实现代码:

    1. package compress
    2. import (
    3. "bytes"
    4. "io"
    5. "github.com/pierrec/lz4"
    6. "github.com/vicanso/elton"
    7. )
    8. const (
    9. // Lz4Encoding lz4 encoding
    10. Lz4Encoding = "lz4"
    11. )
    12. type (
    13. // Lz4Compressor lz4 compress
    14. Lz4Compressor struct {
    15. Level int
    16. MinLength int
    17. }
    18. )
    19. func (l *Lz4Compressor) getMinLength() int {
    20. if l.MinLength == 0 {
    21. return defaultCompressMinLength
    22. }
    23. return l.MinLength
    24. }
    25. // Accept check accept encoding
    26. func (l *Lz4Compressor) Accept(c *elton.Context, bodySize int) (acceptable bool, encoding string) {
    27. // 如果数据少于最低压缩长度,则不压缩
    28. if bodySize >= 0 && bodySize < l.getMinLength() {
    29. return
    30. }
    31. return AcceptEncoding(c, Lz4Encoding)
    32. }
    33. // Compress lz4 compress
    34. func (l *Lz4Compressor) Compress(buf []byte) (*bytes.Buffer, error) {
    35. buffer := new(bytes.Buffer)
    36. w := lz4.NewWriter(buffer)
    37. defer w.Close()
    38. w.Header.CompressionLevel = l.Level
    39. _, err := w.Write(buf)
    40. if err != nil {
    41. return nil, err
    42. }
    43. return buffer, nil
    44. }
    45. // Pipe lz4 pipe compress
    46. func (l *Lz4Compressor) Pipe(c *elton.Context) (err error) {
    47. r := c.Body.(io.Reader)
    48. closer, ok := c.Body.(io.Closer)
    49. if ok {
    50. defer closer.Close()
    51. }
    52. w := lz4.NewWriter(c.Response)
    53. w.Header.CompressionLevel = l.Level
    54. defer w.Close()
    55. _, err = io.Copy(w, r)
    56. return
    57. }

    下面调用示例:

    1. package main
    2. import (
    3. "bytes"
    4. "github.com/vicanso/elton"
    5. compress "github.com/vicanso/elton-compress"
    6. )
    7. func main() {
    8. d := elton.New()
    9. conf := compress.Config{}
    10. lz4 := &compress.Lz4Compressor{
    11. Level: 2,
    12. MinLength: 1024,
    13. }
    14. conf.AddCompressor(lz4)
    15. d.Use(compress.New(conf))
    16. d.GET("/", func(c *elton.Context) (err error) {
    17. b := new(bytes.Buffer)
    18. for i := 0; i < 1000; i++ {
    19. b.WriteString("Hello, World!\n")
    20. }
    21. c.SetHeader(elton.HeaderContentType, "text/plain; charset=utf-8")
    22. c.BodyBuffer = b
    23. return
    24. })
    25. err := d.ListenAndServe(":3000")
    26. if err != nil {
    27. panic(err)
    28. }
    29. }
    1. curl -H 'Accept-Encoding:lz4' -v 'http://127.0.0.1:3000'
    2. * Rebuilt URL to: http://127.0.0.1:3000/
    3. * Trying 127.0.0.1...
    4. * TCP_NODELAY set
    5. * Connected to 127.0.0.1 (127.0.0.1) port 3000 (#0)
    6. > GET / HTTP/1.1
    7. > Host: 127.0.0.1:3000
    8. > User-Agent: curl/7.54.0
    9. > Accept: */*
    10. > Accept-Encoding:lz4
    11. >
    12. < HTTP/1.1 200 OK
    13. < Content-Encoding: lz4
    14. < Content-Length: 103
    15. < Content-Type: text/plain; charset=utf-8
    16. < Vary: Accept-Encoding
    17. ...

    从响应头中可以看出,数据已经压缩为lz4的格式,数据长度仅为103字节,节约了带宽。