• 9. 用Rust手把手编写一个wmproxy(代理,内网穿透等), HTTP2改造篇之HPACK示例, 了解http2头信息如何处理


    9. 用Rust手把手编写一个wmproxy(代理,内网穿透等), HTTP2改造篇之HPACK示例, 了解http2头信息如何处理

    项目 ++wmproxy++

    gite: https://gitee.com/tickbh/wmproxy

    github: https://github.com/tickbh/wmproxy

    关于HPACK相关数据的示例

    长度编码的示例,用5位的前缀示例

    • 将10进行编码,10小于2^5-1,故
    0 1 2 3 4 5 6 7
    +---+---+---+---+---+---+---+---+
    | X | X | X | 0 | 1 | 0 | 1 | 0 | 10 stored on 5 bits
    +---+---+---+---+---+---+---+---+
    • 将1337进行编码
    1. 1337大于2^5-1,故前5位填充
    0 1 2 3 4 5 6 7
    +---+---+---+---+---+---+---+---+
    | X | X | X | 1 | 1 | 1 | 1 | 1 | 31
    +---+---+---+---+---+---+---+---+
    1. 1337 - 31 = 1306,大于128,故需要二次填充,用1306 mod 128 = 21,首位填充1,故8位填充为,当前偏移值为7
    0 1 2 3 4 5 6 7
    +---+---+---+---+---+---+---+---+
    | 1 | 0 | 0 | 1 | 1 | 0 | 1 | 0 | 26
    +---+---+---+---+---+---+---+---+
    1. 对1301-21=1280,1280 / 128 = 10, 10 < 128,故已经完成,首位填0
    0 1 2 3 4 5 6 7
    +---+---+---+---+---+---+---+---+
    | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 10
    +---+---+---+---+---+---+---+---+

    最终的填充值:

    0 1 2 3 4 5 6 7
    +---+---+---+---+---+---+---+---+
    | X | X | X | 1 | 1 | 1 | 1 | 1 | Prefix = 31, I = 1306
    | 1 | 0 | 0 | 1 | 1 | 0 | 1 | 0 | 1306>=128, encode(154), I=1306/128
    | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 0 | 10<128, encode(10), done
    +---+---+---+---+---+---+---+---+

    头部编码示例

    在静态列表中索引

    以下待索引的值

    :method: GET

    十六进制表示值82,二进制表示10000010,表示取静态表2的值,查表为(:method, GET)

    不在列表中,但请求索引,未使用HUFFMAN

    以下示例

    custom-key: custom-header

    十六进制表示

    400a 6375 7374 6f6d 2d6b 6579 0d63 7573 | @.custom-key.cus
    746f 6d2d 6865 6164 6572 | tom-header

    解码过程

    40 | == 01开头请求索引 ==
    0a | name (长度 10)
    6375 7374 6f6d 2d6b 6579 | custom-key
    0d | value (长度 13)
    6375 7374 6f6d 2d68 6561 6465 72 | custom-header
    | -> custom-key:
    | custom-header

    动态表 (解码之后):

    [ 1] (占用 55) custom-key: custom-header
    占用长度: 10+13+32=55

    名字在列表中,但不索引,未使用HUFFMAN

    以下示例

    :path: /sample/path

    十六进制表示

    040c 2f73 616d 706c 652f 7061 7468 | ../sample/path

    解码过程

    04 | == 0000开头,请求不索引 ==
    | name从索引取 (idx = 4)
    | 值为:path
    0c | value (长度12)
    2f73 616d 706c 652f 7061 7468 | /sample/path
    | -> :path: /sample/path

    永不索引,未使用HUFFMAN

    以下示例

    password: secret

    十六进制表示

    1008 7061 7373 776f 7264 0673 6563 7265 | ..password.secre
    74 | t

    解码过程

    10 | == 0001开头不索引 ==
    08 | name (长度8)
    7061 7373 776f 7264 | password
    06 | value (长度6)
    7365 6372 6574 | secret
    | -> password: secret

    完整的请求示例,不使用HUFFMAN

    以下几个示例将连接请求,后续的会用到前面的动态列表

    第一次请求. 示例如下

    :method: GET
    :scheme: http
    :path: /
    :authority: www.example.com

    十六进制表示

    8286 8441 0f77 7777 2e65 7861 6d70 6c65 | ...A.www.example
    2e63 6f6d | .com

    解码过程

    82 | == Indexed - 静态表 ==
    | idx = 2
    | -> :method: GET
    86 | == Indexed - 静态表 ==
    | idx = 6
    | -> :scheme: http
    84 | == Indexed - 静态表 ==
    | idx = 4
    | -> :path: /
    41 | == 01开头请求索引 indexed ==
    | Indexed name (idx = 1)
    | :authority
    0f | Literal value (长度15)
    7777 772e 6578 616d 706c 652e 636f 6d | www.example.com
    | -> :authority:
    | www.example.com

    动态列表 (解码后):

    [ 1->62] (s = 57) :authority: www.example.com
    列表长度: 57

    第二次请求. 示例如下,新加了cache-control字段,其它和第一次一样

    :method: GET
    :scheme: http
    :path: /
    :authority: www.example.com
    cache-control: no-cache

    十六进制表示

    8286 84be 5808 6e6f 2d63 6163 6865 | ....X.no-cache

    解码过程

    82 | == Indexed - 静态表 ==
    | idx = 2
    | -> :method: GET
    86 | == Indexed - 静态表 ==
    | idx = 6
    | -> :scheme: http
    84 | == Indexed - 静态表 ==
    | idx = 4
    | -> :path: /
    be | == Indexed - 动态表,索引值62及以上的为动态表 ==
    | idx = 62
    | -> :authority:
    | www.example.com
    58 | == Literal indexed ==
    | Indexed name (idx = 24)
    | cache-control
    08 | Literal value (8)
    6e6f 2d63 6163 6865 | no-cache
    | -> cache-control: no-cache

    动态列表 (解码后):

    [ 1->62] (s = 53) cache-control: no-cache
    [ 2->63] (s = 57) :authority: www.example.com
    总长度: 110

    第三次请求. 示例如下

    :method: GET
    :scheme: https
    :path: /index.html
    :authority: www.example.com
    custom-key: custom-value

    十六进制表示

    8287 85bf 400a 6375 7374 6f6d 2d6b 6579 | ....@.custom-key
    0c63 7573 746f 6d2d 7661 6c75 65 | .custom-value

    解码过程

    82 | == Indexed - 静态表 ==
    | idx = 2
    | -> :method: GET
    87 | == Indexed - 静态表 ==
    | idx = 7
    | -> :scheme: https
    85 | == Indexed - 静态表 ==
    | idx = 5
    | -> :path: /index.html
    bf | == Indexed - 动态表 ==
    | idx = 63
    | -> :authority:
    | www.example.com
    40 | == Literal indexed ==
    0a | Literal name (长度10)
    6375 7374 6f6d 2d6b 6579 | custom-key
    0c | Literal value (长度12)
    6375 7374 6f6d 2d76 616c 7565 | custom-value
    | -> custom-key:
    | custom-value

    动态列表 (解码后):

    [ 1->62] (s = 54) custom-key: custom-value
    [ 2->63] (s = 53) cache-control: no-cache
    [ 3->64] (s = 57) :authority: www.example.com
    总长度: 164

    完整的请求示例(和上述例子一模一样,但是使用HUFFMAN)

    以下几个示例将连接请求,后续的会用到前面的动态列表

    第一次请求. 示例如下

    :method: GET
    :scheme: http
    :path: /
    :authority: www.example.com

    十六进制表示

    8286 8441 8cf1 e3c2 e5f2 3a6b a0ab 90f4 | ...A......:k....
    ff | .

    比之前少了3字节

    解码过程

    82 | == Indexed - 静态表 ==
    | idx = 2
    | -> :method: GET
    86 | == Indexed - 静态表 ==
    | idx = 6
    | -> :scheme: http
    84 | == Indexed - 静态表 ==
    | idx = 4
    | -> :path: /
    41 | == Literal indexed ==
    | Indexed name (idx = 1)
    | :authority
    8c | Literal value (长度12)
    | Huffman encoded:
    f1e3 c2e5 f23a 6ba0 ab90 f4ff | .....:k.....
    | Decoded:
    | www.example.com
    | -> :authority:
    | www.example.com

    动态列表 (解码后):

    [ 1->62] (s = 57) :authority: www.example.com
    列表长度: 57

    第二次请求. 示例如下,新加了cache-control字段,其它和第一次一样

    :method: GET
    :scheme: http
    :path: /
    :authority: www.example.com
    cache-control: no-cache

    十六进制表示

    8286 84be 5886 a8eb 1064 9cbf | ....X....d..

    比之前少了2字节

    解码过程

    82 | == Indexed - 静态表 ==
    | idx = 2
    | -> :method: GET
    86 | == Indexed - 静态表 ==
    | idx = 6
    | -> :scheme: http
    84 | == Indexed - 静态表 ==
    | idx = 4
    | -> :path: /
    be | == Indexed - 动态表 ==
    | idx = 62
    | -> :authority:
    | www.example.com
    58 | == Literal indexed ==
    | Indexed name (idx = 24)
    | cache-control
    86 | Literal value (长度6)
    | Huffman encoded:
    a8eb 1064 9cbf | ...d..
    | Decoded:
    | no-cache
    | -> cache-control: no-cache

    动态列表 (解码后):

    [ 1->62] (s = 53) cache-control: no-cache
    [ 2->63] (s = 57) :authority: www.example.com
    列表长度: 110

    第三次请求. 示例如下

    :method: GET
    :scheme: https
    :path: /index.html
    :authority: www.example.com
    custom-key: custom-value

    十六进制表示

    8287 85bf 4088 25a8 49e9 5ba9 7d7f 8925 | ....@.%.I.[.}..%
    a849 e95b b8e8 b4bf | .I.[....

    比之前少了5字节

    解码过程

    82 | == Indexed - 静态表 ==
    | idx = 2
    | -> :method: GET
    87 | == Indexed - 静态表 ==
    | idx = 7
    | -> :scheme: https
    85 | == Indexed - 静态表 ==
    | idx = 5
    | -> :path: /index.html
    bf | == Indexed - 动态表 ==
    | idx = 63
    | -> :authority:
    | www.example.com
    40 | == Literal indexed ==
    88 | Literal name (长度8)
    | Huffman encoded:
    25a8 49e9 5ba9 7d7f | %.I.[.}.
    | Decoded:
    | custom-key
    89 | Literal value (长度9)
    | Huffman encoded:
    25a8 49e9 5bb8 e8b4 bf | %.I.[....
    | Decoded:
    | custom-value
    | -> custom-key:
    | custom-value

    动态列表 (解码后):

    [ 1->62] (s = 54) custom-key: custom-value
    [ 2->63] (s = 53) cache-control: no-cache
    [ 3->64] (s = 57) :authority: www.example.com
    总长度: 164

    HUFFMAN编码在于首次如果数据较大的时候优势会更加明显,如果数据较小,或者在后续的时候与普通编码命中索引时基本一致。

    完整的返回示例(HUFFMAN)

    HUFFMAN与普通的差别在于字符串编解码时的差别,这里只介绍一种,并且设置SETTINGS_HEADER_TABLE_SIZE为256

    以下几个示例将连接请求,后续的会用到前面的动态列表

    第一次返回. 示例如下

    :status: 302
    cache-control: private
    date: Mon, 21 Oct 2013 20:13:21 GMT
    location: https://www.example.com

    十六进制表示

    4882 6402 5885 aec3 771a 4b61 96d0 7abe | H.d.X...w.Ka..z.
    9410 54d4 44a8 2005 9504 0b81 66e0 82a6 | ..T.D. .....f...
    2d1b ff6e 919d 29ad 1718 63c7 8f0b 97c8 | -..n..)...c.....
    e9ae 82ae 43d3 | ....C.

    解码过程

    48 | == Literal indexed ==
    | Indexed name (idx = 8)
    | :status
    82 | Literal value (长度2)
    | Huffman encoded:
    6402 | d.
    | Decoded:
    | 302
    | -> :status: 302
    58 | == Literal indexed ==
    | Indexed name (idx = 24)
    | cache-control
    85 | Literal value (长度5)
    | Huffman encoded:
    aec3 771a 4b | ..w.K
    | Decoded:
    | private
    | -> cache-control: private
    61 | == Literal indexed ==
    | Indexed name (idx = 33)
    | date
    96 | Literal value (长度22)
    | Huffman encoded:
    d07a be94 1054 d444 a820 0595 040b 8166 | .z...T.D. .....f
    e082 a62d 1bff | ...-..
    | Decoded:
    | Mon, 21 Oct 2013 20:13:21
    | GMT
    | -> date: Mon, 21 Oct 2013
    | 20:13:21 GMT
    6e | == Literal indexed ==
    | Indexed name (idx = 46)
    | location
    91 | Literal value (长度17)
    | Huffman encoded:
    9d29 ad17 1863 c78f 0b97 c8e9 ae82 ae43 | .)...c.........C
    d3 | .
    | Decoded:
    | https://www.example.com
    | -> location:
    | https://www.example.com

    动态列表 (解码后):

    [ 1->62] (s = 63) location: https://www.example.com
    [ 2->63] (s = 65) date: Mon, 21 Oct 2013 20:13:21 GMT
    [ 3->64] (s = 52) cache-control: private
    [ 4->65] (s = 42) :status: 302
    Table size: 222

    第二次请求. 示例如下,只是状态码发生了变更

    :status: 307
    cache-control: private
    date: Mon, 21 Oct 2013 20:13:21 GMT
    location: https://www.example.com

    十六进制表示

    4883 640e ffc1 c0bf | H.d.....

    解码过程

    48 | == Literal indexed ==
    | Indexed name (idx = 8)
    | :status
    83 | Literal value (长度3)
    | Huffman encoded:
    640e ff | d..
    | Decoded:
    | 307
    | - evict: :status: 302
    | -> :status: 307
    c1 | == Indexed - Add ==
    | idx = 65
    | -> cache-control: private
    c0 | == Indexed - Add ==
    | idx = 64
    | -> date: Mon, 21 Oct 2013
    | 20:13:21 GMT
    bf | == Indexed - Add ==
    | idx = 63
    | -> location:
    | https://www.example.com

    动态列表 (解码后):

    [ 1->62] (s = 42) :status: 307
    [ 2->63] (s = 63) location: https://www.example.com
    [ 3->64] (s = 65) date: Mon, 21 Oct 2013 20:13:21 GMT
    [ 4->65] (s = 52) cache-control: private
    Table size: 222

    由于(:status, 302)的长度为42,且42+222=264>256,所以舍弃最大值

    第三次请求. 示例如下

    :status: 200
    cache-control: private
    date: Mon, 21 Oct 2013 20:13:22 GMT
    location: https://www.example.com
    content-encoding: gzip
    set-cookie: foo=ASDJKHQKBZXOQWEOPIUAXQWEOIU; max-age=3600; version=1

    十六进制表示

    88c1 6196 d07a be94 1054 d444 a820 0595 | ..a..z...T.D. ..
    040b 8166 e084 a62d 1bff c05a 839b d9ab | ...f...-...Z....
    77ad 94e7 821d d7f2 e6c7 b335 dfdf cd5b | w..........5...[
    3960 d5af 2708 7f36 72c1 ab27 0fb5 291f | 9`..'..6r..'..).
    9587 3160 65c0 03ed 4ee5 b106 3d50 07 | ..1`e...N...=P.

    比之前少了5字节

    解码过程

    88 | == Indexed - 静态表 ==
    | idx = 8
    | -> :status: 200
    c1 | == Indexed - 动态表 ==
    | idx = 65
    | -> cache-control: private
    61 | == Literal indexed ==
    | Indexed name (idx = 33)
    | date
    96 | Literal value (长度22)
    | Huffman encoded:
    d07a be94 1054 d444 a820 0595 040b 8166 | .z...T.D. .....f
    e084 a62d 1bff | ...-..
    | Decoded:
    | Mon, 21 Oct 2013 20:13:22
    | GMT
    | - evict: cache-control:
    | private
    | -> date: Mon, 21 Oct 2013
    | 20:13:22 GMT
    c0 | == Indexed - Add ==
    | idx = 64
    | -> location:
    | https://www.example.com
    5a | == Literal indexed ==
    | Indexed name (idx = 26)
    | content-encoding
    83 | Literal value (长度3)
    | Huffman encoded:
    9bd9 ab | ...
    | Decoded:
    | gzip
    | - evict: date: Mon, 21 Oct
    | 2013 20:13:21 GMT
    | -> content-encoding: gzip
    77 | == Literal indexed ==
    | Indexed name (idx = 55)
    | set-cookie
    ad | Literal value (长度45)
    | Huffman encoded:
    94e7 821d d7f2 e6c7 b335 dfdf cd5b 3960 | .........5...[9`
    d5af 2708 7f36 72c1 ab27 0fb5 291f 9587 | ..'..6r..'..)...
    3160 65c0 03ed 4ee5 b106 3d50 07 | 1`e...N...=P.
    | Decoded:
    | foo=ASDJKHQKBZXOQWEOPIUAXQ
    | WEOIU; max-age=3600; versi
    | on=1
    | - evict: location:
    | https://www.example.com
    | - evict: :status: 307
    | -> set-cookie: foo=ASDJKHQ
    | KBZXOQWEOPIUAXQWEOIU; ma
    | x-age=3600; version=1

    动态列表 (解码后):

    [ 1->62] (s = 98) set-cookie: foo=ASDJKHQKBZXOQWEOPIUAXQWEOIU;
    max-age=3600; version=1
    [ 2->63] (s = 52) content-encoding: gzip
    [ 3->64] (s = 65) date: Mon, 21 Oct 2013 20:13:22 GMT
    总长度: 215

    动态列表保留着一个最大的缓存大小值,每一个键值对的计算为name的字节数+value的字节数+32为确定的大小值。超出大小部分则丢弃不缓存,默认大小为4096。

    总结

    HPACK管理着HTTP2的头部的协议部分,有着高压缩比和重复请求的高复用性,双方编码解码需要各自维持一份动态表,动态根据处理数据来动态拓展,保证双方维持的表一模一样。从而保证ID索引不会乱。Huffman编码把头里面需要用到字符串的数据进行进一步的压缩,相对来说整个过程复杂度比HTTP1高很多,但相对的对使用者完全透明,在不影响其使用的情况下提高传输效率,并减少带宽的使用量。

  • 相关阅读:
    Java面试题16-线程池的底层工作原理
    XCTF1-web Robots
    C++多态
    介绍一下mysql有哪些索引类型
    Android Studio 中MotinLayout的简单使用
    使用Python的requests库与chatGPT进行通信
    Leetcode—226.翻转二叉树【简单】
    Rancher 系列文章-Rancher 升级
    封装react-antd-table组件参数以及方法如rowSelection、pageNum、pageSize、分页方法等等
    通过java代码在指定目录生成一个指定内容的vue文件
  • 原文地址:https://www.cnblogs.com/luojiawaf/p/17750621.html