• openresty lua-resty-lock数据库锁

    openresty lua-resty-lock数据库锁

               

    官网：https://github.com/openresty/lua-resty-lock

               

                    

                                              

    lua-resty-lock 说明

               

    1. This library implements a simple mutex lock in a similar way to 
    2. ngx_proxy module's proxy_cache_lock directive.
    3. * lua-resty-lock锁为互斥锁
    5. Under the hood, this library uses ngx_lua module's shared memory 
    6. dictionaries. The lock waiting is nonblocking because we use 
    7. stepwise ngx.sleep to poll the lock periodically
    8. * 使用ngx_lua的共享模块实现锁
    9. * 锁非阻塞，使用ngx.sleep等待，周期性地获取锁状态

            

    new：创建锁

    1. 语法格式：obj, err = lock:new(dict_name, opts?)
    2.  
    3. Creates a new lock object instance by specifying the shared dictionary 
    4. name (created by lua_shared_dict) and an optional options table opts.
    5. * 使用共享空间实现锁，opts参数可选
    6.  
    7. In case of failure, returns nil and a string describing the error.
    8. * 如果失败，返回nil、错误描述信息
    9.  
    10. # opts可选参数
    11. * exptime：锁过期时间，默认30s，可精确到0.001s
    12. * timeout：等待获取锁的超时时间，默认5s，设置为0表示如果获取不到锁，不等待直接离开
    13. * step：锁初始等待时间，单位为秒，可精确到0.001s
    14. * ratio：锁等待时间的放大比例，默认2，每次等待获取锁sleep时长变为上次2倍
    15. * max_step：最大sleep时长，默认0.5s

                      

    lock：加锁

    1. 语法格式：elapsed, err = obj:lock(key)
    2.  
    3. Tries to lock a key across all the Nginx worker processes in the 
    4. current Nginx server instance. Different keys are different locks.
    5. The length of the key string must not be larger than 65535 bytes.
    6. * 所有的worker进程获取同一把锁
    7. * 不同的key表示不同的锁
    8. * key的长度不能大于65535字节
    9.  
    10. Returns the waiting time (in seconds) if the lock is successfully acquired. Otherwise returns nil and a string describing the error.
    11. * 如果加锁成功，返回等待时间
    12. * 加锁失败，返回nil、错误描述信息
    13.  
    14. The waiting time is not from the wallclock, but rather is from simply 
    15. adding up all the waiting "steps". A nonzero elapsed return value indicates 
    16. that someone else has just hold this lock. But a zero return value cannot 
    17. gurantee that no one else has just acquired and released the lock.
    18. * 等待时间是所有steps的和
    19. * 非0值表示有其他worker持有锁
    20. * 0不一定表示又其他worker持有锁，并刚刚释放锁
    21.  
    22. When this method is waiting on fetching the lock, no operating system 
    23. threads will be blocked and the current Lua "light thread" will be 
    24. automatically yielded behind the scene.
    25. * 等待获取锁期间，操作系统线程不会阻塞
    26. * 当前的lua ligth thread会自动让出cpu
    27.  
    28. It is strongly recommended to always call the unlock() method to 
    29. actively release the lock as soon as possible.
    30. * 建议释放锁的时候手动调用unlock
    31.  
    32. If the unlock() method is never called after this method call, the 
    33. lock will get released when
    34.     the current resty.lock object instance is collected automatically by the Lua GC.
    35.     the exptime for the lock entry is reached.
    36. * 如果没有手动调用unlock，锁会在以下情况自动释放
    37.  * lua回收resty.lock对象
    38.  * lock锁过期
    39.  
    40. Common errors for this method call is
    41.     "timeout" : The timeout threshold specified by the timeout option of the new method is exceeded.
    42.     "locked" : The current resty.lock object instance is already holding a lock (not necessarily of the same key).
    43. * 常见错误：锁等待超时、当前对象已经持有锁
    44.  
    45. Other possible errors are from ngx_lua's shared dictionary API.
    46. * 其他可能的错误是ngx_lua的错误
    48. It is required to create different resty.lock instances for multiple 
    49. simultaneous locks (i.e., those around different keys)
    50. * 需要为并发锁（不同的线程）创建不同的锁对象（new创建新的锁对象）

            

    unlock：释放锁

    1. 语法格式：ok, err = obj:unlock()
    2.  
    3. Releases the lock held by the current resty.lock object instance.
    4. Returns 1 on success. Returns nil and a string describing the error otherwise.
    5. * 释放当前锁对象持有的锁
    6. * 如果成功，返回1
    7. * 如果失败，返回nil、错误描述信息
    8.  
    9. If you call unlock when no lock is currently held, the error 
    10. "unlocked" will be returned
    11. * 如果当前对象不持有锁，调用unlock会返回unlocked错误信息

               

    expire：设置当前锁的到期时间

    1. 语法格式：ok, err = obj:expire(timeout)
    2.  
    3. Sets the TTL of the lock held by the current resty.lock object instance. 
    4. This will reset the timeout of the lock to timeout seconds if it is given, 
    5. otherwise the timeout provided while calling new will be used.
    6. * 设置当前锁的到期时间
    7.  
    8. Note that the timeout supplied inside this function is independent from 
    9. the timeout provided while calling new. Calling expire() will not change 
    10. the timeout value specified inside new and subsequent expire(nil) call 
    11. will still use the timeout number from new.
    12. * 如果设置为expire(nil)，锁过期时间仍为new创建的过期时间
    13.  
    14. Returns true on success. Returns nil and a string describing the error otherwise.
    15. * 设置成功，返回true
    16. * 设置失败，返回nil、错误描述信息
    17.  
    18. If you call expire when no lock is currently held, the error 
    19. "unlocked" will be returned
    20. * 如果当前没有持有锁，返回unlocked错误描述信息

              

                       

                                              

    注意事项

               

    为不同的线程创建不同的锁对象

    1. It is always a bad idea to share a single resty.lock object instance 
    2. across multiple ngx_lua "light threads" because the object itself is 
    3. stateful and is vulnerable to race conditions. It is highly recommended 
    4. to always allocate a separate resty.lock object instance for each 
    5. "light thread" that needs one
    6. * 推荐为不同的线程创建不同的锁对象

               

    缓存后端数据，加锁流程

    1. One common use case for this library is avoid the so-called "dog-pile effect", 
    2. that is, to limit concurrent backend queries for the same key when a cache miss 
    3. happens. This usage is similar to the standard ngx_proxy module's proxy_cache_lock 
    4. directive.
    5. * lock锁通常用在缓存时，给后端数据库加锁
    6.  
    7. The basic workflow for a cache lock is as follows:
    8.    * Check the cache for a hit with the key. If a cache miss happens, 
    9.      proceed to step 2.
    10.    * Instantiate a resty.lock object, call the lock method on the key, 
    11.      and check the 1st return value, i.e., the lock waiting time. If it 
    12.      is nil, handle the error; otherwise proceed to step 3.
    13.    * Check the cache again for a hit. If it is still a miss, proceed to step 4; 
    14.      otherwise release the lock by calling unlock and then return the cached value.
    15.    * Query the backend (the data source) for the value, put the result into the 
    16.      cache, and then release the lock currently held by calling unlock
    17. * 一般流程如下
    18.  * 检查缓存是否存在，如果缓存不存在，执行步骤2
    19.  * 创建锁，尝试加锁，获取锁后，执行步骤3
    20.  * 再次检查缓存，如果有数据，直接返回，如果没有，去后端查询数据
    21.  * 后端返回数据，将数据缓存，并返回给客户端

               

    缓存加锁流程

    1.     local resty_lock = require "resty.lock"
    2.     local cache = ngx.shared.my_cache
    3.  
    4.     -- step 1:
    5.     local val, err = cache:get(key)
    6.     if val then
    7.         ngx.say("result: ", val)
    8.         return
    9.     end
    10.  
    11.     if err then
    12.         return fail("failed to get key from shm: ", err)
    13.     end
    14.  
    15.     -- cache miss!
    16.     -- step 2:
    17.     local lock, err = resty_lock:new("my_locks")
    18.     if not lock then
    19.         return fail("failed to create lock: ", err)
    20.     end
    21.  
    22.     local elapsed, err = lock:lock(key)
    23.     if not elapsed then
    24.         return fail("failed to acquire the lock: ", err)
    25.     end
    26.  
    27.     -- lock successfully acquired!
    28.  
    29.     -- step 3:
    30.     -- someone might have already put the value into the cache
    31.     -- so we check it here again:
    32.     val, err = cache:get(key)
    33.     if val then
    34.         local ok, err = lock:unlock()
    35.         if not ok then
    36.             return fail("failed to unlock: ", err)
    37.         end
    38.  
    39.         ngx.say("result: ", val)
    40.         return
    41.     end
    42.  
    43.     --- step 4:
    44.     local val = fetch_redis(key)
    45.     if not val then
    46.         local ok, err = lock:unlock()
    47.         if not ok then
    48.             return fail("failed to unlock: ", err)
    49.         end
    50.  
    51.         -- FIXME: we should handle the backend miss more carefully
    52.         -- here, like inserting a stub value into the cache.
    53.  
    54.         ngx.say("no value found")
    55.         return
    56.     end
    57.  
    58.     -- update the shm cache with the newly fetched value
    59.     local ok, err = cache:set(key, val, 1)
    60.     if not ok then
    61.         local ok, err = lock:unlock()
    62.         if not ok then
    63.             return fail("failed to unlock: ", err)
    64.         end
    65.  
    66.         return fail("failed to update shm cache: ", err)
    67.     end
    68.  
    69.     local ok, err = lock:unlock()
    70.     if not ok then
    71.         return fail("failed to unlock: ", err)
    72.     end
    73.  
    74.     ngx.say("result: ", val)

               

                          

                                              

    使用示例

               

    创建mysql容器

    1. docker run -it -d --net fixed --ip 172.18.0.61 -p 3306:3306 \
    2. -e MYSQL_ROOT_PASSWORD=123456 --name mysql4 mysql

               

    修改数据库：权限、创建表、添加数据

    1. mysql> ALTER USER 'root'@'%' IDENTIFIED WITH mysql_native_password BY '123456';
    2. Query OK, 0 rows affected (0.01 sec)
    3.  
    4. mysql> flush privileges;
    5. Query OK, 0 rows affected (0.00 sec)
    6.  
    7. mysql> create database lihu;
    8. ERROR 1007 (HY000): Can't create database 'lihu'; database exists
    9. mysql> drop database lihu;
    10. Query OK, 1 row affected (0.04 sec)
    11. mysql> create database lihu;
    12. Query OK, 1 row affected (0.01 sec)
    13. mysql> use lihu;
    14. Database changed
    15. mysql> create table test(id int not null primary key auto_increment, name varchar(20));
    16. Query OK, 0 rows affected (0.04 sec)
    17. mysql> insert into test(id, name) values(1, 'gtlx'), (2, 'hzw');
    18. Query OK, 2 rows affected (0.01 sec)
    19. Records: 2  Duplicates: 0  Warnings: 0
    20. mysql> select * from test;
    21. +----+------+
    22. | id | name |
    23. +----+------+
    24. |  1 | gtlx |
    25. |  2 | hzw  |
    26. +----+------+
    27. 2 rows in set (0.00 sec)

                

    nginx.conf

    1. pcre_jit on;
    2.  
    3. #error_log  logs/error.log;
    4. #error_log  logs/error.log  notice;
    5. #error_log  logs/error.log  info;
    6.  
    7. events {
    8.     worker_connections  1024;
    9. }
    10.  
    11.  
    12. http {
    13.     include       mime.types;
    14.     default_type  application/octet-stream;
    15.  
    16.     client_body_temp_path /var/run/openresty/nginx-client-body;
    17.     proxy_temp_path       /var/run/openresty/nginx-proxy;
    18.     fastcgi_temp_path     /var/run/openresty/nginx-fastcgi;
    19.     uwsgi_temp_path       /var/run/openresty/nginx-uwsgi;
    20.     scgi_temp_path        /var/run/openresty/nginx-scgi;
    21.  
    22.     sendfile        on;
    23.  
    24.     keepalive_timeout  65;
    25.  
    26.     include /etc/nginx/conf.d/*.conf;
    27.  
    28.     #设置共享缓存
    29.     lua_shared_dict test 10m;
    30. }

              

    default.conf

    1. server {
    2.     listen       80;
    3.     server_name  localhost;
    4.  
    5.     location / {
    6.         root   /usr/local/openresty/nginx/html;
    7.         index  index.html index.htm;
    8.     }
    9.  
    10.     location /test {
    11.         content_by_lua_block {
    12.             local cache = ngx.shared.test;
    13.  
    14.             local cjson = require 'cjson';
    15.             local resty_lock = require 'resty.lock';
    16.  
    17.             local mysql = require 'resty.mysql';
    18.             local db, err = mysql:new();
    19.             if not db then 
    20.                 ngx.say("mysql创建失败", err);
    21.             end
    22.  
    23.             db:set_timeout(1000);
    24.             local res, err, errcode, sqlstate = db:connect({
    25.                 host = "172.18.0.61", port = 3306, database = "lihu",
    26.                 user = "root", password = "123456"
    27.             });
    28.             if not res then
    29.                 ngx.say("连接出错", err, errcode, sqlstate);
    30.             end
    31.  
    32.             local function fetch_data(id)
    33.                 local res, err, errcode, sqlstate = db:query(
    34.                           "select * from test where id ="..id
    35.                        );
    36.  
    37.                 if not res then
    38.                      ngx.say("数据查询失败", err);
    39.                      return nil;
    40.                 end
    41.  
    42.                 if #res == 0 then
    43.                     ngx.say("后端没有查询到数据");
    44.                     return nil;
    45.                 end
    46.  
    47.                 ngx.say("后端查询结果 ==> ");
    48.                 ngx.say(type(res));
    49.                 ngx.say(cjson.encode(res));
    50.                 for key,value in pairs(res) do
    51.                     ngx.say(key, " ==> ", cjson.encode(value))
    52.                 end
    53.  
    54.                 ngx.say("\n后端返回数据 ==> ",res[1].name); 
    55.                 return res[1].name;
    56.             end
    57.  
    58.             local id = ngx.var.arg_id;
    59.             local value, err = cache:get(id);
    60.             if value then
    61.                 ngx.say("缓存中查询到结果: ", id, " ==> ",  value);
    62.                 return
    63.             end
    64.  
    65.             local lock, err = resty_lock:new("test");
    66.             if not lock then
    67.                 ngx.say("创建锁失败 ==> ", err);
    68.                 return
    69.             end
    70.  
    71.             local elapsed, err = lock:lock("lock");
    72.             if not elapsed then
    73.                 ngx.say("加锁失败 ==> ", err);
    74.                 return
    75.             end
    76.  
    77.             value, err = cache:get(id);
    78.             if value then
    79.                 local ok, err = lock:unlock();
    80.                 if not ok then
    81.                     ngx.say("释放锁失败 ==> ", err);
    82.                 end
    83.  
    84.                 ngx.say("缓存中二次查询获取结果: ", id, " ==> ", value);
    85.                 return
    86.             end
    87.  
    88.             value = fetch_data(id);
    89.             if not value then
    90.                 ok, err = lock:unlock();
    91.                 if not ok then
    92.                     ngx.say("释放锁失败 ==> ", err);
    93.                 end
    94.  
    95.                 ngx.say("数据库没有查询到结果");
    96.                 return
    97.             end
    98.  
    99.             cache:set(id, value);
    100.             ok, err = lock:unlock();
    101.             if not ok then 
    102.                 ngx.say("释放锁失败 ==> ", err);
    103.             end
    104.  
    105.             ngx.say("数据库查询后返回数据: ", id, " ==> ", value);
    106.         }
    107.     }
    108.  
    109.     error_page   500 502 503 504  /50x.html;
    110.     location = /50x.html {
    111.         root   /usr/local/openresty/nginx/html;
    112.     }
    113.  
    114. }

            

    创建openresty容器

    1. docker run -it -d --net fixed --ip 172.18.0.101 -p 8001:80 \
    2. -v /Users/huli/lua/openresty/cache/nginx.conf:/usr/local/openresty/nginx/conf/nginx.conf \
    3. -v /Users/huli/lua/openresty/cache/default.conf:/etc/nginx/conf.d/default.conf \
    4. --name open-cache lihu12344/openresty

             

    使用测试

    1. # 缓存中没有，需要从后端数据库读取数据
    2. huli@hudeMacBook-Pro cache % curl --location --request GET 'localhost:8001/test?id=2'
    3. 后端查询结果 ==> 
    4. table
    5. [{"id":2,"name":"hzw"}]
    6. 1 ==> {"id":2,"name":"hzw"}
    7.  
    8. 后端返回数据 ==> hzw
    9. 数据库查询后返回数据: 2 ==> hzw
    10.  
    11.  
    12. # 再次读取数据时，直接从缓存中获取
    13. huli@hudeMacBook-Pro cache % curl --location --request GET 'localhost:8001/test?id=2'
    14. 缓存中查询到结果: 2 ==> hzw
    15.

               

                       

  • 相关阅读:
    如何自动获取短信验证码？
    了解一下RabbitMQ
    Java -- this关键字
    ZooKeeper~ZooKeeper集群搭建
    AtCoder Beginner Contest 276「A」「B」「C」「D 思维」「E 联通块」「F 树状数组维护期望」
    918. 环形子数组的最大和
    使用JS简单实现一下apply、call和bind方法
    自己最好的canny实现（c#)
    设计与人工智能的关系,人工智能和建筑设计
    【图像处理与机器视觉】图像处理概述与像素
  • 原文地址：https://blog.csdn.net/weixin_43931625/article/details/125918046