强化学习中Sarsa与Q-learning的收敛性和最优性比较（策略类型对比）

1. Sarsa 与Q-learning

Q-learning greedily acquires the optimal state-action value function in each TD step, whereas Sarsa employs an e-greedy strategy to select the next state-action value function within each TD step. In such scenarios, Q-learning tends to preferentially identify an optimal policy, while Sarsa may converge to a suboptimal policy due to its random selection of th