• （6）点云数据处理学习——RGBD图

    1、主要参考

    （1）官网的地址

    RGBD images — Open3D 0.16.0 documentation

    2、实现和测试

     2.1 功能说明

    Open3D有一个用于图像的数据结构。它支持各种函数，如read_image、write_image、filter_image和draw_geometries。一个Open3D图像可以直接转换为numpy数组。

    一个Open3D RGBDImage由两个图像rgbimage .depth和rgbimage .color组成。我们要求两个图像注册到相同的相机帧和具有相同的分辨率。下面的教程展示了如何从许多著名的RGBD数据集中读取和使用RGBD图像。

    2.2Redwood 数据集（Redwood dataset） 

    （1）数据集说明

    在本节中，我们将展示如何从Redwood数据集[Choi2015]_读取和可视化RGBDImage。

    Redwood格式在16位单通道图像中存储深度。整数值表示以毫米为单位的深度测量。它是Open3D解析深度图像的默认格式。

    （2）使用的函数

    create_from_color_and_depth(  color_raw, depth_raw)

    功能说明：默认的转换函数create_rgbd_image_from_color_and_depth从一对彩色和深度图像创建一个RGBDImage。彩色图像被转换为灰度图像，存储在浮动范围[0,1]中。深度图像存储在float中，以米为单位表示深度值。

    （3）测试代码

    1. import open3d as o3d
    2. import numpy as np
    3. from matplotlib import pyplot as plt
    4.  
    5.  
    6. print("Read Redwood dataset")
    7. redwood_rgbd = o3d.data.SampleRedwoodRGBDImages()
    8. color_raw = o3d.io.read_image(redwood_rgbd.color_paths[0])
    9. depth_raw = o3d.io.read_image(redwood_rgbd.depth_paths[0])
    10. rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(
    11.     color_raw, depth_raw)
    12. print(rgbd_image)

    注意open3d自动下载解压后在下面位置

     几张椅子的图挺漂亮的

    2.2.1显示一下RGB图和深度图

    （1）可以通过np.asarray直接对深度图和彩色图进行处理

    1. color_img = np.asarray(rgbd_image.color)
    2. depth_img = np.asarray(rgbd_image.depth)
    3. print(np.size(color_img))
    4. print(color_img[0][0])
    5. print(depth_img[0][0])

    （2）测试代码

    1. import open3d as o3d
    2. import numpy as np
    3. from matplotlib import pyplot as plt
    4.  
    5.  
    6. print("Read Redwood dataset")
    7. redwood_rgbd = o3d.data.SampleRedwoodRGBDImages()
    8. color_raw = o3d.io.read_image(redwood_rgbd.color_paths[0])
    9. depth_raw = o3d.io.read_image(redwood_rgbd.depth_paths[0])
    10. rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(
    11.     color_raw, depth_raw)
    12. print(rgbd_image)
    13.  
    14. #640*480的图
    15. color_img = np.asarray(rgbd_image.color)
    16. depth_img = np.asarray(rgbd_image.depth)
    17. print(np.size(color_img))
    18. print(color_img[0][0])
    19. print(depth_img[0][0])
    20.  
    21.  
    22. plt.subplot(1, 2, 1)
    23. plt.title('Redwood grayscale image')
    24. plt.imshow(rgbd_image.color)
    25. plt.subplot(1, 2, 2)
    26. plt.title('Redwood depth image')
    27. plt.imshow(rgbd_image.depth)
    28. plt.show()

    （3）测试的结果

    2.2.2RGBD图像可以转换为点云图

    1. import open3d as o3d
    2. import numpy as np
    3. from matplotlib import pyplot as plt
    4.  
    5.  
    6. print("Read Redwood dataset")
    7. redwood_rgbd = o3d.data.SampleRedwoodRGBDImages()
    8. color_raw = o3d.io.read_image(redwood_rgbd.color_paths[0])
    9. depth_raw = o3d.io.read_image(redwood_rgbd.depth_paths[0])
    10. rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(
    11.     color_raw, depth_raw)
    12. print(rgbd_image)
    13.  
    14. #640*480的图
    15. # color_img = np.asarray(rgbd_image.color)
    16. # depth_img = np.asarray(rgbd_image.depth)
    17. # print(np.size(color_img))
    18. # print(color_img[0][0])
    19. # print(depth_img[0][0])
    20.  
    21.  
    22. # plt.subplot(1, 2, 1)
    23. # plt.title('Redwood grayscale image')
    24. # plt.imshow(rgbd_image.color)
    25. # plt.subplot(1, 2, 2)
    26. # plt.title('Redwood depth image')
    27. # plt.imshow(rgbd_image.depth)
    28. # plt.show()
    29.  
    30. pcd = o3d.geometry.PointCloud.create_from_rgbd_image(
    31.     rgbd_image,
    32.     o3d.camera.PinholeCameraIntrinsic(
    33.         o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault))
    34. # Flip it, otherwise the pointcloud will be upside down
    35. # pcd.transform([[1, 0, 0, 0], [0, -1, 0, 0], [0, 0, -1, 0], [0, 0, 0, 1]])
    36. # o3d.visualization.draw_geometries([pcd], zoom=0.5)
    37.  
    38. o3d.visualization.draw_geometries([pcd])

     这里使用了PinholeCameraIntrinsicParameters。PrimeSenseDefault作为默认相机参数。它的图像分辨率为640x480，焦距(fx, fy) =(525.0, 525.0)，光心(cx, cy) =(319.5, 239.5)。单位矩阵被用作默认的外部参数。pcd.transform在点云上应用上下翻转转换，以达到更好的可视化目的。

    2.3 SUN数据集(SUN dataset)

     （1）说明

    在本节中，我们将展示如何读取和可视化SUN数据集[Song2015]_的RGBDImage。

    本教程与上面处理Redwood数据集的教程几乎相同。唯一的区别是我们使用了转换函数create_rgbd_image_from_sun_format来解析SUN数据集中的深度图像。

    （2）测试

    1）下载后的数据集如下图所示

     2）测试代码

    1. import open3d as o3d
    2. import numpy as np
    3. from matplotlib import pyplot as plt
    4.  
    5.  
    6.  
    7. print("Read SUN dataset")
    8. sun_rgbd = o3d.data.SampleSUNRGBDImage()
    9. color_raw = o3d.io.read_image(sun_rgbd.color_path)
    10. depth_raw = o3d.io.read_image(sun_rgbd.depth_path)
    11. rgbd_image = o3d.geometry.RGBDImage.create_from_sun_format(color_raw, depth_raw)
    12. print(rgbd_image)
    13.  
    14.  
    15. plt.subplot(1, 2, 1)
    16. plt.title('SUN grayscale image')
    17. plt.imshow(rgbd_image.color)
    18. plt.subplot(1, 2, 2)
    19. plt.title('SUN depth image')
    20. plt.imshow(rgbd_image.depth)
    21. plt.show()
    22.  
    23. pcd = o3d.geometry.PointCloud.create_from_rgbd_image(
    24.     rgbd_image,
    25.     o3d.camera.PinholeCameraIntrinsic(
    26.         o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault))
    27. # Flip it, otherwise the pointcloud will be upside down
    28. # pcd.transform([[1, 0, 0, 0], [0, -1, 0, 0], [0, 0, -1, 0], [0, 0, 0, 1]])
    29. # o3d.visualization.draw_geometries([pcd], zoom=0.5)
    30. o3d.visualization.draw_geometries([pcd])

     转变成的点云图如下图所示

    2.4 NYU数据集(NYU dataset)

    本部分展示如何从NYU数据集[Silberman2012]_读取和可视化RGBDImage。

    （1）说明 

    本部分教程与上面处理Redwood数据集的教程几乎相同，但有两点不同。首先，纽约大学的图片不是标准的jpg或png格式。因此，我们使用mpimg。imread将彩色图像作为numpy数组读取，并将其转换为Open3D图像。调用另一个辅助函数read_nyu_pgm从NYU数据集中使用的特殊大端pgm格式中读取深度图像。其次，我们使用一个不同的转换函数create_rgbd_image_from_nyu_format来解析SUN数据集中的深度图像

    （2）测试代码

    1. import open3d as o3d
    2. import numpy as np
    3.  
    4. import matplotlib.image as mpimg
    5. import re
    6.  
    7.  
    8. # This is special function used for reading NYU pgm format
    9. # as it is written in big endian byte order.
    10. def read_nyu_pgm(filename, byteorder='>'):
    11.     with open(filename, 'rb') as f:
    12.         buffer = f.read()
    13.     try:
    14.         header, width, height, maxval = re.search(
    15.             b"(^P5\s(?:\s*#.*[\r\n])*"
    16.             b"(\d+)\s(?:\s*#.*[\r\n])*"
    17.             b"(\d+)\s(?:\s*#.*[\r\n])*"
    18.             b"(\d+)\s(?:\s*#.*[\r\n]\s)*)", buffer).groups()
    19.     except AttributeError:
    20.         raise ValueError("Not a raw PGM file: '%s'" % filename)
    21.     img = np.frombuffer(buffer,
    22.                         dtype=byteorder + 'u2',
    23.                         count=int(width) * int(height),
    24.                         offset=len(header)).reshape((int(height), int(width)))
    25.     img_out = img.astype('u2')
    26.     return img_out
    27.  
    28.  
    29. print("Read NYU dataset")
    30. # Open3D does not support ppm/pgm file yet. Not using o3d.io.read_image here.
    31. # MathplotImage having some ISSUE with NYU pgm file. Not using imread for pgm.
    32. nyu_rgbd = o3d.data.SampleNYURGBDImage()
    33. color_raw = mpimg.imread(nyu_rgbd.color_path)
    34. depth_raw = read_nyu_pgm(nyu_rgbd.depth_path)
    35. color = o3d.geometry.Image(color_raw)
    36. depth = o3d.geometry.Image(depth_raw)
    37. rgbd_image = o3d.geometry.RGBDImage.create_from_nyu_format(color, depth)
    38. print(rgbd_image)

    下载解压后的图片如下图所示

     （3）显示一下

    1. import open3d as o3d
    2. import numpy as np
    3. from matplotlib import pyplot as plt
    4. import matplotlib.image as mpimg
    5. import re
    6.  
    7.  
    8. # This is special function used for reading NYU pgm format
    9. # as it is written in big endian byte order.
    10. def read_nyu_pgm(filename, byteorder='>'):
    11.     with open(filename, 'rb') as f:
    12.         buffer = f.read()
    13.     try:
    14.         header, width, height, maxval = re.search(
    15.             b"(^P5\s(?:\s*#.*[\r\n])*"
    16.             b"(\d+)\s(?:\s*#.*[\r\n])*"
    17.             b"(\d+)\s(?:\s*#.*[\r\n])*"
    18.             b"(\d+)\s(?:\s*#.*[\r\n]\s)*)", buffer).groups()
    19.     except AttributeError:
    20.         raise ValueError("Not a raw PGM file: '%s'" % filename)
    21.     img = np.frombuffer(buffer,
    22.                         dtype=byteorder + 'u2',
    23.                         count=int(width) * int(height),
    24.                         offset=len(header)).reshape((int(height), int(width)))
    25.     img_out = img.astype('u2')
    26.     return img_out
    27.  
    28.  
    29. print("Read NYU dataset")
    30. # Open3D does not support ppm/pgm file yet. Not using o3d.io.read_image here.
    31. # MathplotImage having some ISSUE with NYU pgm file. Not using imread for pgm.
    32. nyu_rgbd = o3d.data.SampleNYURGBDImage()
    33. color_raw = mpimg.imread(nyu_rgbd.color_path)
    34. depth_raw = read_nyu_pgm(nyu_rgbd.depth_path)
    35. color = o3d.geometry.Image(color_raw)
    36. depth = o3d.geometry.Image(depth_raw)
    37. rgbd_image = o3d.geometry.RGBDImage.create_from_nyu_format(color, depth)
    38. print(rgbd_image)
    39.  
    40. plt.subplot(1, 2, 1)
    41. plt.title('NYU grayscale image')
    42. plt.imshow(rgbd_image.color)
    43. plt.subplot(1, 2, 2)
    44. plt.title('NYU depth image')
    45. plt.imshow(rgbd_image.depth)
    46. plt.show()

     （4）转换为点云图像显示

    1. import open3d as o3d
    2. import numpy as np
    3. from matplotlib import pyplot as plt
    4. import matplotlib.image as mpimg
    5. import re
    6.  
    7.  
    8. # This is special function used for reading NYU pgm format
    9. # as it is written in big endian byte order.
    10. def read_nyu_pgm(filename, byteorder='>'):
    11.     with open(filename, 'rb') as f:
    12.         buffer = f.read()
    13.     try:
    14.         header, width, height, maxval = re.search(
    15.             b"(^P5\s(?:\s*#.*[\r\n])*"
    16.             b"(\d+)\s(?:\s*#.*[\r\n])*"
    17.             b"(\d+)\s(?:\s*#.*[\r\n])*"
    18.             b"(\d+)\s(?:\s*#.*[\r\n]\s)*)", buffer).groups()
    19.     except AttributeError:
    20.         raise ValueError("Not a raw PGM file: '%s'" % filename)
    21.     img = np.frombuffer(buffer,
    22.                         dtype=byteorder + 'u2',
    23.                         count=int(width) * int(height),
    24.                         offset=len(header)).reshape((int(height), int(width)))
    25.     img_out = img.astype('u2')
    26.     return img_out
    27.  
    28.  
    29. print("Read NYU dataset")
    30. # Open3D does not support ppm/pgm file yet. Not using o3d.io.read_image here.
    31. # MathplotImage having some ISSUE with NYU pgm file. Not using imread for pgm.
    32. nyu_rgbd = o3d.data.SampleNYURGBDImage()
    33. color_raw = mpimg.imread(nyu_rgbd.color_path)
    34. depth_raw = read_nyu_pgm(nyu_rgbd.depth_path)
    35. color = o3d.geometry.Image(color_raw)
    36. depth = o3d.geometry.Image(depth_raw)
    37. rgbd_image = o3d.geometry.RGBDImage.create_from_nyu_format(color, depth)
    38. print(rgbd_image)
    39.  
    40. plt.subplot(1, 2, 1)
    41. plt.title('NYU grayscale image')
    42. plt.imshow(rgbd_image.color)
    43. plt.subplot(1, 2, 2)
    44. plt.title('NYU depth image')
    45. plt.imshow(rgbd_image.depth)
    46. plt.show()
    47.  
    48.  
    49. pcd = o3d.geometry.PointCloud.create_from_rgbd_image(
    50.     rgbd_image,
    51.     o3d.camera.PinholeCameraIntrinsic(
    52.         o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault))
    53. # Flip it, otherwise the pointcloud will be upside down
    54. # pcd.transform([[1, 0, 0, 0], [0, -1, 0, 0], [0, 0, -1, 0], [0, 0, 0, 1]])
    55. # o3d.visualization.draw_geometries([pcd], zoom=0.5)
    56. o3d.visualization.draw_geometries([pcd])

    2.5 TUM数据集(TUM dataset)

    （1）说明

    本节展示如何从TUM数据集[Strum2012]_读取和可视化RGBDImage。

    本教程与上面处理Redwood数据集的教程几乎相同。唯一的区别是我们使用转换函数create_rgbd_image_from_tum_format来解析TUM数据集中的深度图像。

    （2）测试

    注意：官网教程有误

    # tum_rgbd = o3d.data.SampleSUNRGBDImage()

    tum_rgbd = o3d.data.SampleTUMRGBDImage()

    或者手动下载

     https://github.com/isl-org/open3d_downloads/releases/download/20220201-data/SampleTUMRGBDImage.zip 

    1. import open3d as o3d
    2. import numpy as np
    3. from matplotlib import pyplot as plt
    4.  
    5.  
    6. print("Read TUM dataset")
    7. # 方法一：官网下载
    8. #tum_rgbd = o3d.data.SampleTUMRGBDImage()
    9. # color_raw = o3d.io.read_image(tum_rgbd.color_path)
    10. # depth_raw = o3d.io.read_image(tum_rgbd.depth_path)
    11. ## 方法二 本地下载后使用 https://github.com/isl-org/open3d_downloads/releases/download/20220201-data/SampleTUMRGBDImage.zip 
    12. tum_rgbd_color_path = "D:/RGBD_CAMERA/python_3d_process/SampleTUMRGBDImage/TUM_color.png"
    13. tum_rgbd_depth_path = "D:/RGBD_CAMERA/python_3d_process/SampleTUMRGBDImage/TUM_depth.png"
    14. color_raw = o3d.io.read_image(tum_rgbd_color_path)
    15. depth_raw = o3d.io.read_image(tum_rgbd_depth_path)
    16. rgbd_image = o3d.geometry.RGBDImage.create_from_tum_format(color_raw, depth_raw)
    17. print(rgbd_image)
    18.  
    19. plt.subplot(1, 2, 1)
    20. plt.title('TUM grayscale image')
    21. plt.imshow(rgbd_image.color)
    22. plt.subplot(1, 2, 2)
    23. plt.title('TUM depth image')
    24. plt.imshow(rgbd_image.depth)
    25. plt.show()
    26.  
    27. pcd = o3d.geometry.PointCloud.create_from_rgbd_image(
    28.     rgbd_image,
    29.     o3d.camera.PinholeCameraIntrinsic(
    30.         o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault))
    31. # # Flip it, otherwise the pointcloud will be upside down
    32. # pcd.transform([[1, 0, 0, 0], [0, -1, 0, 0], [0, 0, -1, 0], [0, 0, 0, 1]])
    33. # o3d.visualization.draw_geometries([pcd], zoom=0.35)
    34. o3d.visualization.draw_geometries([pcd])

     转换后的点云图像如下

     

     

     

  • 相关阅读:
    JSP页面中page指令contentPage/pageEncoding具有什么功能呢？
    Java扫描区块链的工具包｜Java扫块｜监听token转账
    站群服务器的特性和好处是什么
    双11预售在即，小红书品牌如何高效分析竞品？
    微信小程序scroll-view真机出现滚动条如何隐藏
    【动手深度学习-笔记】注意力机制（一）注意力机制框架
    【设计模式从青铜到王者】第四篇:创建型模式
    BacNet4j-跨网段-项目运用
    前端编程应该了解的数据结构——栈、队列
    AIR32F103(六) ADC,I2S,DMA和ADPCM实现的录音播放功能
  • 原文地址：https://blog.csdn.net/chencaw/article/details/128142976