• Java 诊断工具之 Arthas


    简介

    Arthas 是 Alibaba 开源的 Java 诊断工具。Ta 可以动态跟踪 Java 代码,实时监控 JVM 状态,可以在不中断程序执行的情况下轻松完成 JVM 相关问题排查工作 。支持 JDK 6+,支持 Linux/Mac/Windows。

    安装+启动

    1、获取 Arthas

    wget https://alibaba.github.io/arthas/arthas-boot.jar
    
    • 1

    2、启动 Arthas

    java -jar arthas-boot.jar
    
    • 1

    成功启动后的效果,如下图:在这里插入图片描述

    常用命令

    1、stack

    输出当前方法被调用的调用路径。很多时候我们都知道一个方法被执行,但是有很多地方调用了它,你并不知道是谁调用了它,此时你需要的是 stack 命令。

    参数名称参数说明
    class-pattern类名表达式匹配
    method-pattern方法名表达式匹配

    举个例子,我们想要查看 com.annoroad.alpha.api.controller.RestfulController 类中 detail 方法的调用路径,如下图:
    在这里插入图片描述

    2、jad

    反编译指定已加载类的源码。我们可以通过该命令来确定正在运行的代码是不是最新的代码(因为有的时候,会出现代码发布后,代码没有按预期运行的情况)。

    参数名称参数说明
    class类名

    举个例子,如下:

    [arthas@6]$ jad com.annoroad.alpha.api.controller.RestfulController
    
    ClassLoader:
    +-org.springframework.boot.loader.LaunchedURLClassLoader@610db97e
      +-sun.misc.Launcher$AppClassLoader@18b4aac2
        +-sun.misc.Launcher$ExtClassLoader@b3b3114
    
    Location:
    file:/server.jar!/BOOT-INF/classes!/
    
           /*
            * Decompiled with CFR.
            *
            * Could not load the following classes:
            *  com.annoroad.alpha.facade.RestfulFacade
            *  com.annoroad.alpha.facade.UserDto
            *  org.slf4j.Logger
            *  org.slf4j.LoggerFactory
            *  org.springframework.web.bind.annotation.RestController
            */
           package com.annoroad.alpha.api.controller;
    
           import com.annoroad.alpha.api.controller.RestfulController$auxiliary$0NOrXhbf;
           import com.annoroad.alpha.api.controller.RestfulController$auxiliary$95OB8z09;
           import com.annoroad.alpha.api.controller.RestfulController$auxiliary$q2nfGdza;
           import com.annoroad.alpha.api.controller.RestfulController$auxiliary$r7KmaM8n;
           import com.annoroad.alpha.api.controller.RestfulController$auxiliary$rxE4q1Uu;
           import com.annoroad.alpha.api.controller.RestfulController$auxiliary$tipZJ4o0;
           import com.annoroad.alpha.facade.RestfulFacade;
           import com.annoroad.alpha.facade.UserDto;
           import java.lang.reflect.Method;
           import org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.ConstructorInter;
           import org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance;
           import org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter;
           import org.slf4j.Logger;
           import org.slf4j.LoggerFactory;
           import org.springframework.web.bind.annotation.RestController;
    
           @RestController
           public class RestfulController
           implements RestfulFacade,
           EnhancedInstance {
               private static final Logger log;
               private volatile Object _$EnhancedClassField_ws;
               public static volatile /* synthetic */ InstMethodsInter delegate$o54pli0;
               public static volatile /* synthetic */ InstMethodsInter delegate$ggj24u1;
               public static volatile /* synthetic */ ConstructorInter delegate$lpt4b51;
               private static final /* synthetic */ Method cachedValue$U6ueDwbK$4q9ppp1;
               private static final /* synthetic */ Method cachedValue$U6ueDwbK$4pkrst0;
               private static final /* synthetic */ Method cachedValue$U6ueDwbK$0iphd33;
               private static final /* synthetic */ Method cachedValue$U6ueDwbK$b5d02o1;
               private static final /* synthetic */ Method cachedValue$U6ueDwbK$e1rasf2;
    
               public RestfulController() {
                   this(null);
                   delegate$lpt4b51.intercept(this, new Object[0]);
               }
    
               private /* synthetic */ RestfulController(auxiliary.0NOrXhbf nOrXhbf) {
               }
    
               public UserDto detail(long l) {
                   return (UserDto)delegate$o54pli0.intercept(this, new Object[]{l}, new RestfulController$auxiliary$tipZJ4o0(this, l), cachedValue$U6ueDwbK$e1rasf2);
               }
    
               private /* synthetic */ UserDto detail$original$XaXm2T3V(long id) {
    /*18*/         log.info("id : {}", (Object)id);
    /*19*/         return UserDto.builder().build();
               }
    
               public UserDto query(String string, String string2) {
                   return (UserDto)delegate$o54pli0.intercept(this, new Object[]{string, string2}, new RestfulController$auxiliary$95OB8z09(this, string, string2), cachedValue$U6ueDwbK$b5d02o1);
               }
    
               private /* synthetic */ UserDto query$original$XaXm2T3V(String loginName, String userName) {
    /*24*/         log.info("loginName: {}, userName: {}", (Object)loginName, (Object)userName);
    /*25*/         return UserDto.builder().build();
               }
    
               public UserDto create(UserDto userDto) {
                   return (UserDto)delegate$o54pli0.intercept(this, new Object[]{userDto}, new RestfulController$auxiliary$q2nfGdza(this, userDto), cachedValue$U6ueDwbK$0iphd33);
               }
    
               private /* synthetic */ UserDto create$original$XaXm2T3V(UserDto userDto) {
    /*30*/         log.info("userDto : {}", (Object)userDto);
    /*31*/         return userDto;
               }
    
               public UserDto modify(long l, UserDto userDto) {
                   return (UserDto)delegate$o54pli0.intercept(this, new Object[]{l, userDto}, new RestfulController$auxiliary$rxE4q1Uu(this, l, userDto), cachedValue$U6ueDwbK$4pkrst0);
               }
    
               private /* synthetic */ UserDto modify$original$XaXm2T3V(long id, UserDto userDto) {
    /*36*/         log.info("id : {}", (Object)id);
    /*37*/         log.info("userDto : {}", (Object)userDto);
    /*38*/         return userDto;
               }
    
               public void close(long l) {
                   delegate$o54pli0.intercept(this, new Object[]{l}, new RestfulController$auxiliary$r7KmaM8n(this, l), cachedValue$U6ueDwbK$4q9ppp1);
               }
    
               private /* synthetic */ void close$original$XaXm2T3V(long id) {
    /*43*/         log.info("id : {}", (Object)id);
               }
    
               /*
                * Enabled aggressive block sorting
                */
               static {
                   ClassLoader.getSystemClassLoader().loadClass("org.apache.skywalking.apm.dependencies.net.bytebuddy.dynamic.Nexus").getMethod("initialize", Class.class, Integer.TYPE).invoke(null, RestfulController.class, -745233939);
                   cachedValue$U6ueDwbK$4q9ppp1 = RestfulController.class.getMethod("close", Long.TYPE);
                   cachedValue$U6ueDwbK$4pkrst0 = RestfulController.class.getMethod("modify", Long.TYPE, UserDto.class);
                   cachedValue$U6ueDwbK$0iphd33 = RestfulController.class.getMethod("create", UserDto.class);
                   cachedValue$U6ueDwbK$b5d02o1 = RestfulController.class.getMethod("query", String.class, String.class);
                   cachedValue$U6ueDwbK$e1rasf2 = RestfulController.class.getMethod("detail", Long.TYPE);
    /*14*/         log = LoggerFactory.getLogger(RestfulController.class);
               }
    
               @Override
               public Object getSkyWalkingDynamicField() {
                   return this._$EnhancedClassField_ws;
               }
    
               @Override
               public void setSkyWalkingDynamicField(Object object) {
                   this._$EnhancedClassField_ws = object;
               }
    
               final /* synthetic */ UserDto modify$original$XaXm2T3V$accessor$U6ueDwbK(long l, UserDto userDto) {
                   return this.modify$original$XaXm2T3V(l, userDto);
               }
    
               final /* synthetic */ UserDto query$original$XaXm2T3V$accessor$U6ueDwbK(String string, String string2) {
                   return this.query$original$XaXm2T3V(string, string2);
               }
    
               final /* synthetic */ UserDto create$original$XaXm2T3V$accessor$U6ueDwbK(UserDto userDto) {
                   return this.create$original$XaXm2T3V(userDto);
               }
    
               final /* synthetic */ UserDto detail$original$XaXm2T3V$accessor$U6ueDwbK(long l) {
                   return this.detail$original$XaXm2T3V(l);
               }
    
               final /* synthetic */ void close$original$XaXm2T3V$accessor$U6ueDwbK(long l) {
                   this.close$original$XaXm2T3V(l);
               }
           }
    
    Affect(row-cnt:10) cost in 7334 ms.
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47
    • 48
    • 49
    • 50
    • 51
    • 52
    • 53
    • 54
    • 55
    • 56
    • 57
    • 58
    • 59
    • 60
    • 61
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67
    • 68
    • 69
    • 70
    • 71
    • 72
    • 73
    • 74
    • 75
    • 76
    • 77
    • 78
    • 79
    • 80
    • 81
    • 82
    • 83
    • 84
    • 85
    • 86
    • 87
    • 88
    • 89
    • 90
    • 91
    • 92
    • 93
    • 94
    • 95
    • 96
    • 97
    • 98
    • 99
    • 100
    • 101
    • 102
    • 103
    • 104
    • 105
    • 106
    • 107
    • 108
    • 109
    • 110
    • 111
    • 112
    • 113
    • 114
    • 115
    • 116
    • 117
    • 118
    • 119
    • 120
    • 121
    • 122
    • 123
    • 124
    • 125
    • 126
    • 127
    • 128
    • 129
    • 130
    • 131
    • 132
    • 133
    • 134
    • 135
    • 136
    • 137
    • 138
    • 139
    • 140
    • 141
    • 142
    • 143
    • 144
    • 145
    • 146
    • 147
    • 148
    • 149
    • 150
    • 151

    3、sc

    Search-Class 的简写 ,查看 JVM 已加载的类信息。可以通过该命令来确定类是否被真的加载了(因为有的时候,我们会碰到ClassNotFoundException 或 ClassDefNotFoundException 的异常)。

    参数名称参数说明
    class-pattern类名表达式匹配
    method-pattern方法名表达式匹配
    -d输出当前类的详细信息,包括这个类所加载的原始文件来源、类的声明、加载的ClassLoader等详细信息。如果一个类被多个ClassLoader所加载,则会出现多次

    举个例子,我们想要搜索 com.annoroad.alpha.api.controller.RestfulController,如下:

    [arthas@6]$ sc com.annoroad.alpha.api.controller.RestfulController
    com.annoroad.alpha.api.controller.RestfulController
    com.annoroad.alpha.api.controller.RestfulController$$EnhancerBySpringCGLIB$$226b455c
    Affect(row-cnt:2) cost in 23 ms.
    
    • 1
    • 2
    • 3
    • 4

    搜索详细信息的话,如下:

    [arthas@6]$ sc -d com.annoroad.alpha.api.controller.RestfulController
     class-info        com.annoroad.alpha.api.controller.RestfulController
     code-source       file:/server.jar!/BOOT-INF/classes!/
     name              com.annoroad.alpha.api.controller.RestfulController
     isInterface       false
     isAnnotation      false
     isEnum            false
     isAnonymousClass  false
     isArray           false
     isLocalClass      false
     isMemberClass     false
     isPrimitive       false
     isSynthetic       false
     simple-name       RestfulController
     modifier          public
     annotation        org.springframework.web.bind.annotation.RestController
     interfaces        com.annoroad.alpha.facade.RestfulFacade,org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance
     super-class       +-java.lang.Object
     class-loader      +-org.springframework.boot.loader.LaunchedURLClassLoader@610db97e
                         +-sun.misc.Launcher$AppClassLoader@18b4aac2
                           +-sun.misc.Launcher$ExtClassLoader@b3b3114
     classLoaderHash   610db97e
    
     class-info        com.annoroad.alpha.api.controller.RestfulController$$EnhancerBySpringCGLIB$$226b455c
     code-source       file:/server.jar!/BOOT-INF/classes!/
     name              com.annoroad.alpha.api.controller.RestfulController$$EnhancerBySpringCGLIB$$226b455c
     isInterface       false
     isAnnotation      false
     isEnum            false
     isAnonymousClass  false
     isArray           false
     isLocalClass      false
     isMemberClass     false
     isPrimitive       false
     isSynthetic       false
     simple-name       RestfulController$$EnhancerBySpringCGLIB$$226b455c
     modifier          public
     annotation
     interfaces        org.springframework.aop.SpringProxy,org.springframework.aop.framework.Advised,org.springframework.cglib.proxy.Factory
     super-class       +-com.annoroad.alpha.api.controller.RestfulController
                         +-java.lang.Object
     class-loader      +-org.springframework.boot.loader.LaunchedURLClassLoader@610db97e
                         +-sun.misc.Launcher$AppClassLoader@18b4aac2
                           +-sun.misc.Launcher$ExtClassLoader@b3b3114
     classLoaderHash   610db97e
    
    Affect(row-cnt:2) cost in 23 ms.
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41
    • 42
    • 43
    • 44
    • 45
    • 46
    • 47

    4、sm

    与 sc 类似,sm 是 Search-Method 的简写,查看已加载类的方法信息,如下:

    [arthas@6]$ sm com.annoroad.alpha.api.controller.RestfulController
    com.annoroad.alpha.api.controller.RestfulController$$EnhancerBySpringCGLIB$$226b455c <init>()V
    com.annoroad.alpha.api.controller.RestfulController$$EnhancerBySpringCGLIB$$226b455c getTargetClass()Ljava/lang/Class;
    com.annoroad.alpha.api.controller.RestfulController$$EnhancerBySpringCGLIB$$226b455c getSkyWalkingDynamicField()Ljava/lang/Object;
    com.annoroad.alpha.api.controller.RestfulController$$EnhancerBySpringCGLIB$$226b455c setSkyWalkingDynamicField(Ljava/lang/Object;)V
    com.annoroad.alpha.api.controller.RestfulController$$EnhancerBySpringCGLIB$$226b455c setCallbacks([Lorg/springframework/cglib/proxy/Callback;)V
    com.annoroad.alpha.api.controller.RestfulController$$EnhancerBySpringCGLIB$$226b455c CGLIB$SET_STATIC_CALLBACKS([Lorg/springframework/cglib/proxy/Callback;)V
    com.annoroad.alpha.api.contro
    ......
    Affect(row-cnt:78) cost in 28 ms.
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10

    搜索详细信息的话,如下:

    [arthas@6]$ sm -d com.annoroad.alpha.api.controller.RestfulController
     declaring-class   com.annoroad.alpha.api.controller.RestfulController$$EnhancerBySpringCGLIB$$226b455c
     constructor-name  <init>
     modifier          public
     annotation
     parameters
     exceptions
     classLoaderHash   610db97e
    
     declaring-class  com.annoroad.alpha.api.controller.RestfulController$$EnhancerBySpringCGLIB$$226b455c
     method-name      getTargetClass
     modifier         final,public
     annotation
     parameters
     return           java.lang.Class
     exceptions
     classLoaderHash  610db97e
    
     declaring-class  com.annoroad.alpha.api.controller.RestfulController$$EnhancerBySpringCGLIB$$226b455c
     method-name      getSkyWalkingDynamicField
     modifier         final,public
     annotation
     parameters
     return           java.lang.Object
     exceptions
     classLoaderHash  610db97e
    ......
    Affect(row-cnt:78) cost in 28 ms. 
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28

    5、watch

    可以监测一个方法的入参和返回值。

    参数名称参数说明
    class-pattern类名表达式匹配
    method-pattern方法名表达式匹配
    express观察表达式
    condition-express条件表达式
    -b在方法调用之前观察
    -e在方法异常之后观察
    -s在方法返回之后观察
    -f在方法结束之后(正常返回和异常返回)观察,默认选项
    -E开启正则表达式匹配,默认为通配符匹配
    -x指定输出结果的属性遍历深度,默认为 1

    举个例子,我们要查看 com.annoroad.alpha.api.controller.RestfulController 类中 detail 方法的出参和返回值,如下:

    [arthas@6]$ watch com.annoroad.alpha.api.controller.RestfulController detail {params,returnObj} -x 2
    Press Q or Ctrl+C to abort.
    Affect(class count: 2 , method count: 2) cost in 702 ms, listenerId: 3
    method=com.annoroad.alpha.api.controller.RestfulController.detail location=AtExit
    ts=2022-08-04 13:06:03; [cost=1.920253ms] result=@ArrayList[
        @Object[][
            @Long[1],
        ],
        @UserDto[
            id=null,
            loginName=null,
            userName=null,
            password=null,
            email=null,
            mobile=null,
            status=null,
            emailAuth=null,
            remarks=null,
            createTime=null,
            updateTime=null,
        ],
    ]
    method=com.annoroad.alpha.api.controller.RestfulController$$EnhancerBySpringCGLIB$$226b455c.detail location=AtExit
    ts=2022-08-04 13:06:03; [cost=504.253835ms] result=@ArrayList[
        @Object[][
            @Long[1],
        ],
        @UserDto[
            id=null,
            loginName=null,
            userName=null,
            password=null,
            email=null,
            mobile=null,
            status=null,
            emailAuth=null,
            remarks=null,
            createTime=null,
            updateTime=null,
        ],
    ]
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9
    • 10
    • 11
    • 12
    • 13
    • 14
    • 15
    • 16
    • 17
    • 18
    • 19
    • 20
    • 21
    • 22
    • 23
    • 24
    • 25
    • 26
    • 27
    • 28
    • 29
    • 30
    • 31
    • 32
    • 33
    • 34
    • 35
    • 36
    • 37
    • 38
    • 39
    • 40
    • 41

    如果我们要查看异常的话,可以在 {params,returnObj} 里增加 throwExp ,如:{params,returnObj,throwExp}

    6、trace

    输出方法内部的调用路径,以及路径上每个节点的耗时。

    参数名称参数说明
    class-pattern类名表达式匹配
    method-pattern方法名表达式匹配
    express观察表达式
    condition-express条件表达式
    -E开启正则表达式匹配,默认为通配符匹配
    -n命令执行次数
    #cost方法执行耗时

    举个例子,我们要查看 com.annoroad.alpha.api.controller.RestfulController 类中 detail 方法的调用路径,,以及路径上每个节点的耗时,如下:

    [arthas@6]$ trace com.annoroad.alpha.api.controller.RestfulController detail
    Press Q or Ctrl+C to abort.
    Affect(class count: 2 , method count: 2) cost in 687 ms, listenerId: 7
    `---ts=2022-08-04 13:17:57;thread_name=http-nio-20000-exec-3;id=44;is_daemon=true;priority=5;TCCL=org.springframework.boot.web.embedded.tomcat.TomcatEmbeddedWebappClassLoader@2ac05a33
        `---[9.918521ms] com.annoroad.alpha.api.controller.RestfulController$$EnhancerBySpringCGLIB$$226b455c:detail()
            `---[98.63% 9.782571ms ] org.springframework.cglib.proxy.MethodInterceptor:intercept()
                `---[16.66% 1.629865ms ] com.annoroad.alpha.api.controller.RestfulController:detail()
                    +---[1.57% 0.025631ms ] com.annoroad.alpha.api.controller.RestfulController$auxiliary$tipZJ4o0:<init>()
                    `---[90.97% 1.482733ms ] org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter:intercept()
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9

    如果只想看到耗时大于 1ms 的节点,如下:

    [arthas@6]$ trace com.annoroad.alpha.api.controller.RestfulController detail '#cost > 1'
    Press Q or Ctrl+C to abort.
    Affect(class count: 2 , method count: 2) cost in 632 ms, listenerId: 10
    `---ts=2022-08-04 13:21:56;thread_name=http-nio-20000-exec-1;id=42;is_daemon=true;priority=5;TCCL=org.springframework.boot.web.embedded.tomcat.TomcatEmbeddedWebappClassLoader@2ac05a33
        `---[3.996851ms] com.annoroad.alpha.api.controller.RestfulController$$EnhancerBySpringCGLIB$$226b455c:detail()
            `---[98.23% 3.926263ms ] org.springframework.cglib.proxy.MethodInterceptor:intercept()
                `---[51.64% 2.027492ms ] com.annoroad.alpha.api.controller.RestfulController:detail()
                    +---[0.69% 0.013957ms ] com.annoroad.alpha.api.controller.RestfulController$auxiliary$tipZJ4o0:<init>()
                    `---[94.76% 1.921184ms ] org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.InstMethodsInter:intercept()
    
    • 1
    • 2
    • 3
    • 4
    • 5
    • 6
    • 7
    • 8
    • 9

    7、dashboard

    查看当前系统的实时数据面板,这个命令可以全局的查看 JVM 运行状态,例如:内存 和 CPU 占用情况,如下:
    在这里插入图片描述

    列名说明
    IDJava 级别的线程 ID,注意这个 ID 不能跟 jstack 中的 nativeID 一一对应 我们可以通过 thread id 查看线程的堆栈信息
    NAME线程名
    GROUP线程组名
    PRIORITY线程优先级,1~10之间的数字,越大表示优先级越高
    STATE线程的状态
    CPU%线程消耗的 CPU 占比,采样 100ms,将所有线程在这 100ms 内的 CPU 使用量求和,再算出每个线程的 CPU 使用占比
    TIME线程运行总时间,数据格式为 分:秒
    INTERRUPTED线程当前的中断位状态
    DAEMON是否是 daemon 线程

    8、jobs

    执行后台异步任务。线上有些问题是偶然发生的,这时就需要使用异步任务,把信息写入文件。

    PS:因为还没有用到,所以这里就先不写了,有需要的同学,可以去【参考】中提供的连接地址查看

    9、logger

    查看 logger 信息,更新 logger level。

    PS:因为还没有用到,所以这里就先不写了,有需要的同学,可以去【参考】中提供的连接地址查看

    10、redefine

    redefine jvm 已加载的类 ,可以在不重启项目的情况下,热更新类。

    PS:因为还没有用到,所以这里就先不写了,有需要的同学,可以去【参考】中提供的连接地址查看

    参考

  • 相关阅读:
    如何长期有效维护客户关系,你真的了解你的客户吗?
    DO、DTO、BO、VO、POJO 的区别
    mysql的增删改查
    I350网卡烧录oprom,通过UEFI PXE引导方案
    信创之国产浪潮电脑+统信UOS操作系统体验2:安装visual studio code和cmake搭建C++开发环镜
    数据结构与算法_哈希表_线性探测法原理和代码实现
    C# ZIP解压缩 模拟时钟
    IMAU鸿蒙北向开发-2023年9月6日学习日志
    C++|前言
    删除链表的倒数第n个节点的最优算法实现
  • 原文地址:https://blog.csdn.net/yangchao1125/article/details/126155217