怎么创建公司的个人网站,网站制作方案模板,c# 开发网站开发,临沭做网站Title
题目
Efficient anatomical labeling of pulmonary tree structures via deeppoint-graph representation-based implicit fields
通过深度点图表示的隐式场实现肺树结构的高效解剖标注 01
文献速递介绍
近年来#xff0c;肺部疾病#xff08;Decramer等#xff… Title
题目
Efficient anatomical labeling of pulmonary tree structures via deeppoint-graph representation-based implicit fields
通过深度点图表示的隐式场实现肺树结构的高效解剖标注 01
文献速递介绍
近年来肺部疾病Decramer等2008Marcus等2000Nunes等2017已成为全球死亡的主要原因Quaderi和Hurst2018肺部研究因此受到了越来越多的关注。在与肺部疾病相关的研究中通过医学影像了解肺部解剖结构非常重要因为肺部疾病与从肺部CT图像中推导出的度量指标之间有已知的关联Charbonnier等2019Kirby等2019Qin等2019Shaw等2002Loud等2001Shen等2014。一些研究也强调了拓扑保持和对关键位置的学习Yu等2023Tan等2021Xie等2022。然而从点云或图中提取可用的表面并非易事。
考虑到上述挑战如图2e所示我们希望开发一种优化的自动化解决方案该方案在计算上高效并且能够重建具有连续表面定义和拓扑保持的3D模型特别关注关键结构点。为此我们提出了一种新方法使得骨架图上下文可以深入注入到基于点的表示中并采用隐式方法生成特征场从而实现高效且去噪的密集体积重建。所提出的 隐式点图网络IPGN 包括点和图编码器结合多个 点图融合 层作为 点图网络PGN 进行深度特征融合并使用 隐式点模块 来建模3D中的隐式表面允许在任意位置进行快速分类推理。得益于隐式表示的灵活性经过肺部树标注训练的IPGN可以通过简单修改推理方法第六节进一步扩展到肺部分段重建。
复杂的树状肺部结构——气道、动脉和静脉如图1所示——具有较高的分支因子并在肺部呼吸系统中发挥重要作用。在体积图像中对肺树进行多类语义分割其中每个类别代表根据肺部分段的医学定义划分的特定树枝或分支是建模其复杂性的有效方法挑战主要体现在分叉位置和远端分支Yu等2023。在肺树标注中衍生的定量特征Kirby等2019Smith等2014Shaw等2002不仅与肺部疾病和肺部相关的医学应用有关Loud等2001Shen等2014而且对外科导航至关重要Qin等2019。本研究集中于调查和开发深度学习方法用于高效准确的肺部气道、动脉和静脉的解剖标注。
在深度学习方法中卷积神经网络CNN已成为语义分割的事实标准方法Zhou等20182019。它们的优点之一是能够生成具有明确表面的体积。然而当处理大型3D体积时CNN计算需求较高并且在降低分辨率图2(a)或在局部补丁上操作时图2(b)往往会导致缺乏细节或全球上下文从而得到不令人满意的结果。相比之下点云是一种稀疏的3D形状表示没有可用的表面但其计算需求较低同时能够保持全局结构图2(c)。此外由于目标本身具有树状结构因此图建模图2(d)成为一种有效的方式。 Aastract
摘要
Pulmonary diseases rank prominently among the principal causes of death worldwide. Curing them will require,among other things, a better understanding of the complex 3D tree-shaped structures within the pulmonarysystem, such as airways, arteries, and veins. Traditional approaches using high-resolution image stacks andstandard CNNs on dense voxel grids face challenges in computational efficiency, limited resolution, localcontext, and inadequate preservation of shape topology. Our method addresses these issues by shifting fromdense voxel to sparse point representation, offering better memory efficiency and global context utilization.However, the inherent sparsity in point representation can lead to a loss of crucial connectivity in tree-shapedstructures. To mitigate this, we introduce graph learning on skeletonized structures, incorporating differentiablefeature fusion for improved topology and long-distance context capture. Furthermore, we employ an implicitfunction for efficient conversion of sparse representations into dense reconstructions end-to-end. The proposedmethod not only delivers state-of-the-art performance in labeling accuracy, both overall and at key locations,but also enables efficient inference and the generation of closed surface shapes. Addressing data scarcity inthis field, we have also curated a comprehensive dataset to validate our approach.
肺部疾病在全球范围内位居主要死亡原因之列。治疗这些疾病需要更深入地理解肺部系统内复杂的三维树状结构如气道、动脉和静脉。传统方法依赖于高分辨率图像堆栈和标准卷积神经网络CNN在稠密体素网格上面临计算效率、分辨率、局部上下文以及形状拓扑保持不足等挑战。我们的方法通过从稠密体素表示转向稀疏点表示来解决这些问题提供了更好的内存效率和全局上下文利用。然而点表示的固有稀疏性可能导致树状结构中重要连接性的丧失。为了解决这个问题我们引入了基于骨架结构的图学习结合可微分特征融合以改进拓扑结构和长距离上下文的捕捉。此外我们采用隐式函数来高效地将稀疏表示转换为密集重建进行端到端处理。所提出的方法不仅在标注准确性无论是整体标注还是关键位置的标注上实现了最先进的性能还实现了高效推理和闭合表面形状的生成。为了应对该领域的数据稀缺问题我们还整理了一个全面的数据集以验证我们的方法。 Method
方法
Given a binary volumetric image of a pulmonary tree (Fig. 6(a)), agraph (Fig. 6(b)) is constructed with VesselVio (Bumgarner and Nelson,
from the original volume following the procedure in 3.2.2, anda set of points (6) are randomly sampled from the foreground voxelsto construct a point cloud (Fig. 6(c), referred to as original pointcloud). While the original point cloud is a sparse and downsampledversion of the foreground volume, the graph represents a skeleton ofthe pulmonary tree.We first introduce the general notation rule for both point and graphelements. While the coordinates of points and graph nodes arerepresented as P and G, single point or graph element is expressed as pand , where P {p1 , p2 , …, p}, G {1 , 2 ,… , }. The superscriptnotation p () represents an element’s feature at the th network layer.At input, the 3-dimensional {, , } point coordinates, P ∈ R×3and graph nodes, G ∈ R×3 are utilized as initial feature. We use a pointneural network, and a graph neural network as initial feature encoders,from which we extract a 128-dimensional intermediate feature for eachpoint and graph node, expressed as P (0) ∈ R×128 and G (0) ∈ R×128 .Subsequently, initial features from both branches P (0) and G (0)are incorporated within one or multiple Point-Graph Fusion layers,which allow for two-way feature integration based on feature propagation (Rossi et al., 2021) and ball-querygrouping (Qi et al., 2017).Let the input to a Point-Graph Fusion layer be defined as P (−1) andG (−1), the feature out of the fusion layer is P () and G () . The last PointGraph Fusion layer outputs P () and G (*) after Point-Graph Fusionlayers for deep feature fusion. Finally, a lightweight MLP network anda GNN projects the fusion feature to 19-dimensional vectors for graph(Fig. 6(d)) and point predictions (Fig. 6(e)).An Implicit Point Module is further introduced to reconstruct thedense volumes, which consists of a feature propagation process and anMLP network. As features are extracted by the Point-Graph Network,the Implicit Point Module leverages the extracted multi-stage pointfeatures for fast dense volume segmentation. We define the points thatare not necessarily in the original point cloud, but are in the foregroundof the tree volume, thus requiring classification, as query points. Givena query point p with at arbitrary coordinates, the module locates p ’s-nearest point elements from the original point cloud: {p1 , p2 , … , p },and extracts their multi-stage features {1 , 2 , …, } from the backboneNetwork for feature propagation into a multi-stage representation of the query point p . After propagating the point feature , theMLP network is utilized to make class predictions. By applying thisprocess to all foreground points, we can efficiently generate a densevolume reconstruction (Fig. 6(f)).To avoid naming ambiguity, we refer to the aforementioned complete network as Implicit Point-Graph Network (IPGN), and that sans theimplicit part as the backbone network, Point-Graph Network (PGN).
给定一个肺部树的二值体积图像图6(a)首先使用VesselVioBumgarner和Nelson2022从原始体积中构建一个图图6(b)按照3.2.2节的过程并从前景体素中随机采样一组点6个点来构建点云图6(c)称为原始点云。原始点云是前景体积的稀疏和下采样版本而图则表示肺部树的骨架。
接下来我们首先介绍点和图元素的一般表示法规则。点的坐标和图节点的坐标分别表示为P和G其中单个点或图元素表示为p和并且P {p₁, p₂, …, pₘ}G {₁, ₂, …, ₙ}。上标p()表示元素在第层网络中的特征。
在输入阶段使用三维点坐标{, , }P ∈ Rᵐ×3 和图节点G ∈ Rⁿ×3 作为初始特征。我们使用点神经网络和图神经网络作为初始特征编码器从中提取每个点和图节点的128维中间特征表示为P(0) ∈ Rᵐ×128 和 G(0) ∈ Rⁿ×128。
随后来自两个分支的初始特征P(0) 和 G(0) 被整合到一个或多个点图融合层中允许基于特征传播Rossi等2021和球查询与分组Qi等2017进行双向特征融合。设定点图融合层的输入为P(−1) 和 G(−1)经过融合层后的输出特征为P() 和 G()。最后一个点图融合层在深度特征融合之后输出P() 和 G()。
最终一个轻量级的MLP网络和图神经网络GNN将融合后的特征投影为19维的向量用于图预测图6(d)和点预测图6(e)。
进一步引入了一个隐式点模块Implicit Point Module来重建稠密体积该模块由特征传播过程和MLP网络组成。当通过点图网络提取特征时隐式点模块利用提取的多阶段点特征进行快速的稠密体积分割。我们定义了那些不一定位于原始点云中的点而是位于树体积前景中的点需要分类称为查询点。给定一个查询点pₖ其坐标为任意位置模块会定位到pₖ的个最近邻点元素{p₁, p₂, …, pₖ}并从骨干网络中提取其多阶段特征{₁, ₂, …, ₖ}通过特征传播将其转化为查询点pₖ的多阶段表示ₖ。在传播点特征ₖ之后MLP网络被用来进行类别预测。通过将此过程应用于所有前景点我们可以高效地生成稠密体积重建(f)。
为了避免命名上的歧义我们将上述完整网络称为隐式点图网络IPGN不包括隐式部分的称为骨干网络点图网络PGN。 Conclusion
结论
In conclusion, we take an experimentally comprehensive deep-diveinto pulmonary tree segmentation based on the compiled PTL dataset.A novel architecture Implicit Point-Graph Network (IPGN) is presentedfor accurate and efficient pulmonary tree segmentation. Our methodleverages a dual-branch point-graph fusion model to effectively capturethe complex branching structure of the respiratory system. Extensiveexperiment results demonstrate that by implicit modeling on pointgraph features, the proposed model achieves state-of-the-art segmentation quality with minimum computation cost for practical dense volumereconstruction. The advancements made in this study could potentiallyenhance the diagnosis, management, and treatment of pulmonary diseases, ultimately improving patient outcomes in this critical area ofhealthcare.
总之我们对基于编制的PTL数据集的肺树分割进行了实验性的深入研究。我们提出了一种新型架构——隐式点图网络IPGN用于实现准确高效的肺树分割。我们的方法利用双分支点图融合模型有效捕捉呼吸系统复杂的分支结构。大量实验结果表明通过对点图特征进行隐式建模所提模型在最小计算成本下实现了最先进的分割质量能够进行实用的密集体积重建。本研究取得的进展有望提升肺部疾病的诊断、管理和治疗最终改善患者在这一关键医疗领域的治疗效果。 Figure
图 Fig. 1. Pulmonary Tree Labeling. (a) The pulmonary tree consists of three anatomicstructures (airway, artery and vein). (b) Given a binary volume representing a treestructure as input, we label each voxel into one of 19 classes (18 pulmonary segments1 background out of lung) based on branching regions.
图1. 肺树标注。(a) 肺树由三种解剖结构组成气道、动脉和静脉。(b) 给定一个表示树状结构的二进制体积作为输入我们根据分支区域将每个体素标注为19个类别中的一个18个肺部分段和1个肺外背景。 Fig. 2. A Comparison of Data Representation for Pulmonary Tree Labeling. The CNN-based voxel methods are either low-resolution (down-sampled) or local (sliding-window).The standard sparse representation like point and graph is global but it is not trivial to reconstruct high-quality dense volume. Our method that combines point, graph, and implicitfields produces high-quality dense reconstruction efficiently
图2. 肺树标注数据表示的比较。基于CNN的体素方法要么是低分辨率下采样要么是局部滑动窗口。标准的稀疏表示如点和图是全局的但重建高质量的密集体积并非易事。我们的方法结合了点、图和隐式场能够高效地生成高质量的密集重建。 Fig. 3. Anatomy of Pulmonary Trees and Pulmonary Segments. Each pulmonarytree branch corresponds to a pulmonary segment. The intersegmental vein is highlightedin red, which lies along the pulmonary segment border.
图3. 肺树解剖与肺部分段。每个肺树分支对应一个肺部分段。红色高亮显示的是段间静脉它位于肺部分段边界上 Fig. 4. Sample Illustration with Voxel-level and Graph-level Metrics. Due to thelarge number of voxels, voxel-level metric often overlooks labeling at key regions. Giventwo test samples with similar voxel performance, the sample with better graph-levelperformance (Case A and B) tends to perform better at key points than its counterpart(Case A’ and B’)
图4. 体素级和图级度量的示例说明。由于体素数量庞大体素级度量通常忽视了关键区域的标注。给定两个测试样本它们具有相似的体素表现但图级度量表现更好的样本案例A和B在关键点上的表现往往优于其对应样本案例A和B。 Fig. 5. Skeleton Graphs Directly from Bumgarner and Nelson (2022) and Two Types**of Imperfections. (1) The centerline (CL) points, which represent the path of thetree branch as discrete and connected coordinates, might contain cliques. (2) Thecoordinates of leaf graph nodes might fall outside the foreground volume
图5. 直接来自Bumgarner和Nelson (2022)的骨架图和两种缺陷类型。(1) 中线CL点表示树分支路径的离散且连接的坐标可能包含团簇。(2) 叶图节点的坐标可能位于前景体积之外。 Fig. 6. Overview of the Proposed Implicit Point-Graph Network (IPGN) for Pulmonary Tree Labeling. The pipeline pre-processes dense volume to graph and point cloudinput. The Point-Graph Fusion layers enhance point features with graph context with differentiable feature fusion learning, and the Implicit Point Module produces dense predictionbased on the deep sparse representation efficiently
图6. 提出的隐式点图网络IPGN用于肺树标注的概述。该流程将密集体积预处理为图和点云输入。点图融合层通过可微分特征融合学习增强点特征并结合图上下文隐式点模块基于深度稀疏表示高效地生成密集预测。 Fig. 7. The Detailed Operation of a Point-Graph Fusion layer. The figure only shows the operations for one node, and one point, which are highlighted as Node/Point ofinterest. In reality, this process is parallel for all nodes and points. Here, 1 and 2 are MLP networks while represents a graph neural network.
图7. 点图融合层的详细操作。图中仅显示了一个节点和一个点的操作它们被高亮显示为感兴趣的节点/点。实际上这个过程对所有节点和点是并行进行的。这里1 和 2 是多层感知机MLP网络 表示图神经网络。 Fig. 8. Implicit Point Module. For any query point, the Implicit Point Moduleconsumes multi-stage features from a Point-Graph Network with feature propagationand a neural network to provide a label
图8. 隐式点模块。对于任何查询点隐式点模块通过特征传播和神经网络利用来自点图网络的多阶段特征为该点提供标签。 Fig. 9. Visualization of Segmentation Results. This figure displays segmentation results using GAT (Velickovic et al., 2017) w/ post-processing (Section 5.1.3), PointNet (Qiet al., 2017) and IPGN (with GAT and PointNet backbones) methods along with the ground truth. The initial-form predictions (graph or point cloud) before dense volumeprediction are also presented. Below each dense volume reconstruction, we show voxel (V) dice and node (N) dice results
图9. 分割结果的可视化。此图展示了使用GATVelickovic等2017带后处理第5.1.3节、PointNetQi等2017和IPGN结合GAT和PointNet骨架方法的分割结果以及真实标签。还展示了在密集体积预测之前的初始预测形式图或点云。在每个密集体积重建下方显示了体素V和节点N的Dice系数结果。 Fig. 10. Pulmonary Segments Visualization. A sample reconstruction of pulmonarysegments based on the airway, artery, and vein is presented against ground truth inaxial, sagittal and coronal views.
图10. 肺部分段可视化。展示了基于气道、动脉和静脉的肺部分段重建样本并与真实标签在轴向、矢状面和冠状面视图中进行对比。 Table
表 Table 1Notations and corresponding descriptions. This table presents the notations used to represent point and graph elements within the architecture.The notations are organized into two stages: Point-Graph Feature Fusion and Implicit Dense Volume Reconstruction
表1. 符号及对应描述。此表列出了用于表示架构中点和图元素的符号。符号分为两个阶段点图特征融合和隐式密集体积重建。 Table 2Model performance in dice score at voxel and graph level on the PTL dataset. Baseline methods using different feature representations are presented to compare againstthe proposed methods on the Pulmonary Tree Labeling (PTL) dataset. PGNPoint-Graph Network without implicit fields; IPGNImplicit Point Graph Network; KPkey point.: Graph-level predictions from Voxel/Point Networks are acquired by inferencing on the node and edge locations.
表2PTL数据集上体素级和图级模型性能的Dice系数。展示了使用不同特征表示的基准方法以便与提出的方法进行比较数据集为肺树标注PTL。PGN 无隐式字段的点图网络IPGN 隐式点图网络KP 关键点。从体素/点网络的图级预测通过在节点和边位置进行推理得到。 Table 3Inference speed and segmentation metrics for dense volume reconstruction. This table compares the dense volume segmentation test time and dice score across voxel, point,graph-based, and point-graph fusion methods. The test times are measured in seconds while dice score presents segmentation quality.
表3. 密集体积重建的推理速度和分割指标。此表比较了体素、点、图和点图融合方法在密集体积分割测试中的时间和Dice系数。测试时间以秒为单位Dice系数表示分割质量。 Table 4Impact of input for graph modeling. We show the results using MLP and GAT(Velickovic et al., 2017) backbones on different feature inputs
表4. 输入对图建模的影响。我们展示了在不同特征输入下使用MLP和GATVelickovic等2017骨架的结果。 Table 5Multi-stage point feature input for implicit point module.This table presents thedice results using different combinations of concatenated multi-stage features fromthe backbone network as input to the Point Implicit Module. (Point Encoder: PointTransformer (Zhao et al., 2021); Graph Encoder: GAT (Velickovic et al., 2017)).
表5. 隐式点模块的多阶段点特征输入。此表展示了使用来自骨干网络的不同组合拼接的多阶段特征作为输入到点隐式模块时的Dice系数结果。点编码器PointTransformerZhao等2021图编码器GATVelickovic等2017。 Table 6Model performance on the original/corrupted data. We present experiment resultson the original and corrupted version (Weng et al., 2023) of the PTL dataset for PointTransformer (Zhao et al., 2021) and the proposed IPGN with voxel-level dice score (%)as metric.
表6. 原始数据/损坏数据上的模型性能。据集上进行的实验结果比较了PointTransformerZhao等2021和提出的IPGN模型使用体素级Dice系数%作为指标。 Table 7Model complexity. This table highlights the parameter counts for key competitive methods. The performance is represented as voxel-level dice(%)
表7. 模型复杂度。此表展示了关键竞争方法的参数数量。性能以体素级Dice系数%表示。
