将脑部磁共振成像（MRI）转化为诊断用正电子发射断层扫描（PET）：通过基于注意力的编码器-解码器网络从多对比度MRI合成15O-水PET脑血流量（CBF）。

Oldlee

Title

题目

Turning brain MRI into diagnostic PET: 15O-water PET CBF synthesis from multi-contrast MRI via attention-based encoder–decoder networks

文献速递介绍

脑血管疾病是全球性的公共健康问题，影响着所有种族和民族群体（Yusuf等，2001）。仅中风每年就影响1500万人，导致500万人死亡和500万人永久性残疾，对家庭、社区和医疗系统造成压力（Mukherjee和Patil，2011）。脑血管疾病的早期诊断和正确评估可以减少对大脑的损害并加速治疗。此外，脑血流量（CBF）异常往往与多种神经系统疾病相关，包括血管畸形、癫痫和阿尔茨海默病等神经退行性疾病（Iturria-Medina等，2016；Leijenaar等，2017）。因此，准确的CBF量化对于脑血管疾病的诊断和评估至关重要。

使用放射性水（15O-水）的正电子发射断层扫描（PET）被广泛认为是人类CBF测量的金标准成像技术（Ito等，2004）。然而，由于其高昂的成本、复杂的后勤支持以及电离辐射的使用，PET并不普及，全球只有大约20个中心提供15O-水PET CBF成像，且大多用于研究领域。磁共振成像（MRI）是一种更易获取且具有成本效益的替代方案，其中动脉自旋标记（ASL）灌注MRI和动态磁敏感对比（DSC）灌注MRI是量化CBF最常见的两种检查方式（Detre等，1992；Villringer等，1988）。尽管MRI的应用广泛，但在脑血流量存在全局或局部减少的情况下（常见于脑血管疾病患者），基于MRI的CBF图可能不够准确（Grade等，2015）。这促使了将MRI扫描转化为PET样CBF图像的图像转图像翻译方法的发展，这些方法相比灌注MRI的CBF测量可在定量和定性评估方面提升准确性，并能适用于更广泛的患者群体和指征，而不仅限于PET成像可行的范围。

随着计算机视觉的进步以及医学影像数据库的不断扩大，利用深度学习的图像到图像翻译方法得以发展，能够将一种医学成像模式转换为另一种。例如，从MRI预测计算机断层扫描（CT）图像（Kearney等，2020）、从CT预测MRI（Jin等，2019）、从PET预测CT（Armanious等，2019）、从CT预测PET（Ben-Cohen等，2019）以及从PET预测MRI（Bazangani等，2022）。

Aastract

摘要

Accurate quantification of cerebral blood flow (CBF) is essential for the diagnosis and assessment of a wide range of neurological diseases. Positron emission tomography (PET) with radiolabeled water ( 15O-water) is the gold-standard for the measurement of CBF in humans, however, it is not widely available due to its prohibitive costs and the use of short-lived radiopharmaceutical tracers that require onsite cyclotron production. Magnetic resonance imaging (MRI), in contrast, is more accessible and does not involve ionizing radiation. This study presents a convolutional encoder–decoder network with attention mechanisms to predict the gold-standard15O-water PET CBF from multi-contrast MRI scans, thus eliminating the need for radioactive tracers. The model was trained and validated using 5-fold cross-validation in a group of 126 subjects consisting of healthy controls and cerebrovascular disease patients, all of whom underwent simultaneous 15O-water PET/MRI. The results demonstrate that the model can successfully synthesize high-quality PET CBF measurements (with an average SSIM of 0.924 and PSNR of 38.8 dB) and is more accurate compared to concurrent and previous PET synthesis methods. We also demonstrate the clinical significance of the proposed algorithm by evaluating the agreement for identifying the vascular territories with impaired CBF. Such methods may enable more widespread and accurate CBF evaluation in larger cohorts who cannot undergo PET imaging due to radiation concerns, lack of access, or logistic challenges.

准确量化脑血流量（CBF）对于广泛的神经系统疾病的诊断和评估至关重要。使用放射性水（15O-水）的正电子发射断层扫描（PET）是人体CBF测量的金标准，但由于高昂的成本和需要现场回旋加速器生产的短寿命放射性药物，其应用并不广泛。相比之下，磁共振成像（MRI）更易获取且不涉及电离辐射。本研究提出了一种带有注意力机制的卷积编码器-解码器网络，用于从多对比度MRI扫描中预测金标准的15O-水PET CBF，从而消除了对放射性示踪剂的需求。模型在126名受试者（包括健康对照组和脑血管病患者）组成的群体中使用五折交叉验证进行训练和验证，这些受试者均接受了同步的15O-水PET/MRI。结果表明，该模型能够成功合成高质量的PET CBF测量值（平均结构相似性指数（SSIM）为0.924，峰值信噪比（PSNR）为38.8 dB），其准确性优于当前及以往的PET合成方法。我们还通过评估识别血流量受损的血管区域的一致性，证明了该算法的临床意义。这类方法可能使无法进行PET成像的群体（因辐射、获取限制或后勤障碍）也能广泛且准确地进行CBF评估。

Method

方法

4.1. Proposed network architecture

Fig. 2 shows the architecture of the proposed 3D convolutional encoder–decoder network. The input to the network is an 8-channel tensor ?? ∈ Rℎ×??×??×8 that includes data from structural MRI (T1w, T2-FLAIR) and perfusion-related MRI scans (ASL difference images from SD-ASL and MD-ASL acquisitions, PD images obtained as part of quantification for the SD-ASL CBF calculations), as well as quantified metrics such as SD-CBF, MD-CBF, and ATT derived from MD-ASL. The output of the network ?? ∈ Rℎ×??×??×1 denotes the 15O-water PET CBF map. To enable the transformation of multi-contrast MRI into PET CBF, an attention-based encoder–decoder network is developed to serve as a non-linear mapping function ???? , such that ?? = ???? (??), where ?? contains the network parameters to be learned. In a previous study, we demonstrated the ability to predict the goldstandard 15O-water PET CBF from a set of 16 input MRI contrasts using a 2D convolutional neural network (Guo et al., 2020). In this study, we improved the quality of synthetic PET by utilizing an attention-based 3D structure that capitalized on the spatial information across eight volumetric MRI scans and captured the long-range feature interactions necessary for accurate predictions. However, the application of 3D models to brain image-to-image translation problems is limited by the scarcity of annotated brain imaging data and the associated high computational cost. Therefore, we employed a larger cohort of healthy controls and cerebrovascular disease patients for the current study and applied several data augmentation strategies to further expand the overall number of PET/MRI data samples needed for improved translation performance. This included flipping (horizontally and vertically), shifting (horizontally and vertically), and rotating (clockwise and anti-clockwise) the input and output images, resulting in an eightfold increase in the dataset size. Finally, a custom loss function was carefully designed to maintain contextual and structural information in input multi-contrast MRI scans and thus optimize the performance of the MRI-to-PET translation network.

4.1. 提出网络架构

图2展示了所提出的3D卷积编码器-解码器网络的架构。该网络的输入为8通道张量?? ∈ Rℎ×??×??×8，包括结构性MRI（T1加权、T2-FLAIR）和与灌注相关的MRI扫描（来自SD-ASL和MD-ASL采集的ASL差分图像，作为SD-ASL CBF计算一部分的PD图像），以及从MD-ASL中获得的定量指标（如SD-CBF、MD-CBF和ATT）。网络输出?? ∈ Rℎ×??×??×1表示15O-水PET CBF图像。为实现多对比度MRI到PET CBF的转换，开发了一个基于注意力的编码器-解码器网络，用作非线性映射函数????，使得?? = ????(??)，其中??为需学习的网络参数。

在先前的研究中，我们使用2D卷积神经网络预测了一组16种输入MRI对比度下的金标准15O-水PET CBF（Guo等，2020）。在本研究中，通过使用基于注意力的3D结构，提升了合成PET的质量。该结构利用了8个体积MRI扫描的空间信息，并捕捉到准确预测所需的长距离特征交互。然而，将3D模型应用于脑部图像到图像的翻译问题受限于带注释的脑成像数据的稀缺性以及较高的计算成本。因此，本研究使用了更大的健康对照组和脑血管疾病患者队列，并采用了多种数据增强策略以进一步扩展PET/MRI数据样本总量，从而提升翻译性能。这些策略包括水平和垂直翻转、水平和垂直平移以及顺时针和逆时针旋转输入和输出图像，使数据集大小增加了八倍。最后，精心设计的自定义损失函数用于保持输入多对比度MRI扫描中的上下文和结构信息，从而优化MRI到PET的翻译网络性能。

Conclusion

结论

PET imaging of CBF is a critical component in the diagnosis and assessment of cerebrovascular diseases. However, its use is limited because of its prohibitive cost and the use of ionizing radiation. This study introduces an attention-based convolutional encoder–decoder network for synthesizing 15O-water PET CBF maps from multi-contrast MRI scans without using radioactive tracers. The performance of the proposed image-to-image translation network is examined for different network settings and input MRI sequence combinations. Quantitative evaluations show improved PET synthesis results compared to previous MRI-to-PET CBF prediction models. Additionally, qualitative results also reveal that regional CBF values in synthetic PET are in strong agreement with those of the ground-truth PET, with no statistically significant difference between them. In patients with cerebrovascular diseases, brain regions with abnormally low CBF were accurately identified in synthetic PET CBF maps. This technique has the potential to increase the accessibility of cerebrovascular disease assessment for underserved populations, underprivileged communities, and developing nations, without the need for expensive and radiation-emitting PET imaging.

PET成像在脑血流量（CBF）的诊断和脑血管疾病的评估中起着关键作用。然而，由于成本高昂且涉及电离辐射，其应用受限。本研究提出了一种基于注意力的卷积编码器-解码器网络，用于从多对比度MRI扫描中合成15O-水PET CBF图像，而无需使用放射性示踪剂。研究对该图像到图像的翻译网络在不同网络设置和输入MRI序列组合下的性能进行了评估。定量评价显示，与先前的MRI到PET CBF预测模型相比，PET合成结果有所改善。此外，定性结果也表明合成PET的区域CBF值与真实PET高度一致，二者之间无统计学显著差异。在脑血管疾病患者中，合成的PET CBF图准确识别出CBF异常低的脑区。这一技术有望提高脑血管疾病评估的可及性，尤其适用于资源匮乏的人群、弱势社区和发展中国家，无需昂贵且带有辐射的PET成像。

Figure

图

Fig. 1. Experimental design for measuring CBF using PET/MRI in two cohorts. In cohort 1, three simultaneous PET/ASL acquisitions were acquired from the participants in a single visit (two scans before and one scan 15 min after the administration of the vasodilator [acetazolamide, ACZ]). In cohort 2, two simultaneous PET/ASL acquisitions were acquired from the participants in each visit (one before and one 15 min after ACZ administration); Of the 41 HCs incohort 2, 31 had two separate imaging sessions on different days.

图1. 使用PET/MRI在两个队列中测量脑血流量（CBF）的实验设计。在队列1中，参与者在一次访视中进行了三次同步的PET/ASL采集（两次扫描在扩血管剂[乙酰唑胺, ACZ]给药前进行，一次扫描在给药后15分钟进行）。在队列2中，参与者在每次访视中进行了两次同步的PET/ASL采集（一次在ACZ给药前，一次在给药后15分钟进行）；在队列2的41名健康对照者（HCs）中，有31人在不同的日期进行了两次单独的成像会话。

Fig. 2. Attention-based encoder–decoder network architecture for predicting PET CBF maps from multi-contrast MRI. The input to the network is an 8-channel tensor ?? ∈ R8∶96×96×64that includes data from T1w, T2-FLAIR, PD, SD-ASL, MD-ASL, SD-CBF, MD-CBF, and ATT. 15O-water PET CBF is the target image. The number of channels is shown above each of the encoder and decoder blocks. Conv3D = 3D convolutional layer, GN = group normalization, PReLU = parametric rectified linear unit, Conv3DTranspose = transposed 3D convolution layer, and MaxPooling3D = Max pooling operation for 3D data.

图2. 基于注意力的编码器-解码器网络架构，用于从多对比度MRI预测PET CBF图像。网络的输入为8通道张量?? ∈ R8∶96×96×64，包含T1w、T2-FLAIR、PD、SD-ASL、MD-ASL、SD-CBF、MD-CBF和ATT的数据。15O-水PET CBF为目标图像。每个编码器和解码器模块上方显示通道数。Conv3D = 3D卷积层，GN = 组归一化，PReLU = 参数化线性整流单元，Conv3DTranspose = 转置3D卷积层，MaxPooling3D = 3D数据的最大池化操作。

Fig. 3. The schematic of an attention mechanism used in the 3D convolutional encoder–decoder network. Input features (???? ) are multiplied element-wise by attention coefficients (??) computed in the attention module. The gating features (???? ) collected from a lower layer of the network are used to identify the spatial regions of interest with relevant activations and contextual information, GN denotes group normalization.

图3. 用于3D卷积编码器-解码器网络中的注意力机制示意图。输入特征（????）通过在注意力模块中计算出的注意力系数（??）进行元素级乘法。网络中较低层收集的门控特征（????）用于识别具有相关激活和上下文信息的空间感兴趣区域。GN表示组归一化。

Fig. 4. MRI-to-PET prediction results for healthy control and cerebrovascular disease patients in the axial plane: Examples of input multi-contrast MRI scans, output synthetic PET, reference PET, and corresponding magnified (×3) absolute error maps. PET CBF is quantified in milliliters per 100 g of brain tissue per minute (ml/100 g/min).

图4. 健康对照组和脑血管疾病患者在轴平面上的MRI到PET预测结果：包括输入的多对比度MRI扫描、输出的合成PET、参考PET，以及相应的放大（×3）绝对误差图。PET脑血流量（CBF）以每分钟每100克脑组织的毫升数（ml/100 g/min）进行量化。

Fig. 5. MRI-to-PET prediction results for healthy control and cerebrovascular disease patients in the coronal plane: Examples of input multi-contrast MRI scans, output synthetic PET, reference PET, and corresponding magnified (×3) absolute error maps. PET CBF is quantified in ml/100 g/min.

图5. 健康对照组和脑血管疾病患者在冠状面上的MRI到PET预测结果：包括输入的多对比度MRI扫描、输出的合成PET、参考PET，以及相应的放大（×3）绝对误差图。PET脑血流量（CBF）以每分钟每100克脑组织的毫升数（ml/100 g/min）进行量化。

Fig. 6. Examples of PET CBF prediction (in ml/100 g/min) for healthy controls and cerebrovascular disease patients at different loss functions and network settings: (a) example results of the reference PET CBF against synthetic PET CBF and magnified (×3) absolute error maps at different settings in the axial plane, (b) quantitative comparison between different loss functions and network elements.

图6. 在不同损失函数和网络设置下，健康对照组和脑血管疾病患者的PET脑血流量（CBF，单位：ml/100 g/min）预测示例：（a）在轴平面上，不同设置下参考PET CBF与合成PET CBF及其放大（×3）的绝对误差图的示例结果，（b）不同损失函数和网络元素的定量比较。

Fig. 7. Qualitative comparisons of different deep learning models in synthesizing PET CBF maps from multi-contrast MR images. CBFs are quantified in ml/100 g/min. The proposed method produces more accurate predictions than the standard 3D U-Net, 3D c-GAN, and 3D SC-GAN, particularly for those with abnormal lesions.

图7. 不同深度学习模型在从多对比度MR图像中合成PET脑血流量（CBF）图的质量比较。CBF以每分钟每100克脑组织的毫升数（ml/100 g/min）进行量化。所提出的方法比标准3D U-Net、3D c-GAN和3D SC-GAN的预测更准确，尤其是在存在异常病灶的情况下。

Fig. 8. Bland-Altman plots of the mean CBF in the ASPECTS vascular territories for PreDiamox (top panel) and PostDiamox (bottom panel) measurements. Each panel includes three plots showing the agreement between the reference PET CBF (True PET) and (i) the PET CBF produced by our model (Synthetic PET, left), (ii) CBF derived from single-delay ASL (SD-CBF, middle), and (iii) CBF derived from multi-delay ASL (MD-CBF, right).

图8. ASPECTS血管区域中平均脑血流量（CBF）的Bland-Altman图，分别显示在Diamox使用前（上图）和使用后（下图）的测量结果。每个图包含三幅子图，显示参考PET CBF（真实PET）与以下各项之间的一致性：(i) 我们模型生成的PET CBF（合成PET，左图），(ii) 单延迟ASL获得的CBF（SD-CBF，中图），以及(iii) 多延迟ASL获得的CBF（MD-CBF，右图）。

Fig. 9. Joint plots of mean CBF in ASPECTS vascular territories for PreDiamox (top panel) and PostDiamox (bottom panel) measurements: Each panel displays three plots for the relationship and distribution histogram between True PET and Synthetic PET (left), SD-CBF (middle), and MD-CBF (right). The regression line and Pearson correlation coefficient ® are added to each of the joint plots.

图9. ASPECTS血管区域中平均脑血流量（CBF）的联合图，分别显示Diamox使用前（上图）和使用后（下图）的测量结果：每个图板展示了真实PET与合成PET（左图）、SD-CBF（中图）、以及MD-CBF（右图）之间的关系和分布直方图。每个联合图中均添加了回归线和皮尔森相关系数（r）。

Fig. 10. ROC curves and AUC scores for identifying vascular territories with reduced CBF in Prediamox (top panel) and PostDiamox (bottom panel) measurements for cerebrovascular patients: Each panel includes three plots showing the classification performance at three different threshold values, (i) Threshold at 2 standard deviation (STD) below the mean CBF of healthy control participants (left), (ii) Threshold at 3 STD below mean CBF (middle), and (iii) Threshold at 4 STD below mean CBF (right). Each plot includes three ROC curves showing the classification performance of Synthetic PET (blue curve), SD-CBF (red curve), and MD-CBF (green curve).

图10. 用于识别脑血管病患者中Prediamox（上图）和PostDiamox（下图）测量中脑血流量（CBF）降低的血管区域的ROC曲线和AUC得分：每个图板包含三个不同阈值下的分类性能图，（i）阈值设为健康对照组平均CBF低2个标准差（STD）（左图），（ii）阈值设为平均CBF低3个STD（中图），以及（iii）阈值设为平均CBF低4个STD（右图）。每幅图中均包含三条ROC曲线，分别显示合成PET（蓝色曲线）、SD-CBF（红色曲线）和MD-CBF（绿色曲线）的分类性能。

Fig. 11. Radar charts of classification performance measures for SD-CBF, MD-CBF, and synthetic PET CBF at Threshold = Mean – 3 STD: (a) and (b) Evaluation metrics for detecting abnormal regions (i.e., regions with reduced CBF) in PreDiamox and PostDiamox measurements, respectively.

图11. SD-CBF、MD-CBF和合成PET CBF在阈值 = 平均值 - 3个标准差（STD）下的分类性能雷达图：(a)和(b)分别表示在Prediamox和PostDiamox测量中检测异常区域（即CBF降低区域）的评价指标。

Fig. 12. MRI-to-PET CBF prediction using either ASL data or structural MRI. Each panel illustrates the data of a separate subject, with real axial and coronal images as well as synthetic images and magnified absolute error maps.

图12. 使用ASL数据或结构性MRI进行MRI到PET CBF的预测。每个图板展示了不同受试者的数据，包括真实的轴向和冠状图像、合成图像以及放大的绝对误差图。

Fig. 13. Bland-Altman plots of the mean CBF in True PET and Synthetic PET for healthy controls and cerebrovascular disease patients at PreDiamox (top panel) and PostDiamox (bottom panel) measurements when ASL data is used as an input to our network

图13. 在使用ASL数据作为输入时，真实PET和合成PET中健康对照组和脑血管疾病患者的平均脑血流量（CBF）的Bland-Altman图，分别显示在Prediamox（上图）和PostDiamox（下图）测量中的结果。

Fig. 14. Bland-Altman plots of the mean CBF in True PET and Synthetic PET for healthy controls and cerebrovascular disease patients at PreDiamox (top panel) and PostDiamox (bottom panel) measurements when structural MR data is used as an input to our network.

图14. 在使用结构性MR数据作为输入时，真实PET和合成PET中健康对照组和脑血管疾病患者的平均脑血流量（CBF）的Bland-Altman图，分别显示在Prediamox（上图）和PostDiamox（下图）测量中的结果。

Fig. B.1. Brain arterial vascular territories of ASPECTS.

图B.1. ASPECTS的脑动脉血管区域。

Fig. C.1. Bland-Altman plots of the mean CBF in the ASPECTS vascular territories for healthy controls (HC, top panel) and cerebrovascular disease patients (PT, bottom panel). Each panel includes three plots showing the agreement between the True PET CBF and (i) the Synthetic PET (left), (ii) SD-CBF (middle), and (iii) MD-CBF (right).

图C.1. ASPECTS血管区域中健康对照组（HC，上图）和脑血管疾病患者（PT，下图）的平均脑血流量（CBF）Bland-Altman图。每个图板包含三幅子图，分别显示真实PET CBF与(i)合成PET（左图）、(ii) SD-CBF（中图）和(iii) MD-CBF（右图）之间的一致性。

Fig. D.1. Comparison of PET CBF synthesis performance: Impact of MD-ASL and SD-ASL contrasts.

图D.1. PET CBF合成性能比较：MD-ASL和SD-ASL对比度的影响。

Table

表

Table 1Demographic information of the dataset. Age is presented as mean ± standard deviation.

表1数据集的人口统计信息。年龄以平均值 ± 标准差表示。

Table 2PET synthesis results for healthy controls and cerebrovascular disease patients. The model is evaluated using five-fold cross-validation, and the quantitative metrics are computed for the whole brain region. ↑/↓ denotes that higher/lower values correspond to better quality of synthetic PET. Results are presented as mean ± standard deviation (std).

表2健康对照组和脑血管疾病患者的PET合成结果。模型通过五折交叉验证进行评估，定量指标针对全脑区域计算。↑/↓表示较高/较低的数值对应于更好的合成PET质量。结果以平均值 ± 标准差（std）表示。

Table 3Quantitative comparison between our model and baseline models.

表3我们的模型与基线模型的定量比较。

Table 4PET synthesis error (true PET CBF − synthetic PET CBF): Bias and precision in synthetic PET measurements for healthy controls and patients at different scan times. Mean and SD represent the bias and variability in the measurements, respectively. No. refers to the number of PET/MRI observations.

表4PET合成误差（真实PET CBF − 合成PET CBF）：在不同扫描时间下健康对照组和患者的合成PET测量中的偏差和精度。平均值和标准差（SD）分别表示测量中的偏差和变异性。No.表示PET/MRI观察次数

Table A.1List of parameters used for MRI acquisition.

表A.1MRI采集中使用的参数列表。