Hi! It’s hard to troubleshoot without seeing the code. In terms of changes needed for 03_dqn_play, it should work out of the box, as dqn_n_steps affects only the training samples generation process, but the network architecture is exactly the same.

To troubleshoot, I’d suggest checking the Q-values produced by the network (just printing them out should be fine), maybe, the network is producing weird values or output tensor is treated in a wrong way.

If it won’t help, I’d suggest you to describe you changes in detail on github issues in the repo: https://github.com/PacktPublishing/Deep-Reinforcement-Learning-Hands-On/issues