论文标题
RIS辅助全双工UL和DL传输的DRL方法：波束形成，相移和功率优化
A DRL Approach for RIS-Assisted Full-Duplex UL and DL Transmission: Beamforming, Phase Shift and Power Optimization
论文作者
论文摘要
我们为全双工（FD）传输提出了深入的增强学习方法（DRL）方法，该方法可以预测可重新配置的智能表面（RIS），基站（BS）主动束式轴承的相移以及发射功率，以最大程度地提高上线链路链路和下链接用户的加权总和速率。现有方法需要通道状态信息（CSI）和残留自我解干（SI）知识来计算精确的主动束形式或DRL奖励，这些奖励通常在没有CSI或残留SI的情况下失败。特别是对于随时间变化的通道，需要在每个时间步骤中对DRL代理进行估算和信号CSI，并且代价高昂。我们提出了一个两阶段的DRL框架，其信号传导开销很小，以解决此问题。第一阶段使用最小二乘方法通过部分取消残留SI来启动学习。第二阶段使用DRL实现与现有基于CSI的方法相当的性能，而无需CSI或确切的剩余SI。此外，量化RIS相位的提议的DRL框架使用$ 32 $ $ 32的$ 32 $倍的信号降低了从BS到RISS的信号传导。量化的方法降低了动作空间，从而比连续方法分别降低了收敛速度和$ 7.1 \％$和$ 22.28 \％$ $ $和DL速率。
We propose a deep reinforcement learning (DRL) approach for a full-duplex (FD) transmission that predicts the phase shifts of the reconfigurable intelligent surface (RIS), base station (BS) active beamformers, and the transmit powers to maximize the weighted sum rate of uplink and downlink users. Existing methods require channel state information (CSI) and residual self-interference (SI) knowledge to calculate exact active beamformers or the DRL rewards, which typically fail without CSI or residual SI. Especially for time-varying channels, estimating and signaling CSI to the DRL agent is required at each time step and is costly. We propose a two-stage DRL framework with minimal signaling overhead to address this. The first stage uses the least squares method to initiate learning by partially canceling the residual SI. The second stage uses DRL to achieve performance comparable to existing CSI-based methods without requiring the CSI or the exact residual SI. Further, the proposed DRL framework for quantized RIS phase shifts reduces the signaling from BS to the RISs using $32$ times fewer bits than the continuous version. The quantized methods reduce action space, resulting in faster convergence and $7.1\%$ and $22.28\%$ better UL and DL rates, respectively than the continuous method.
