Understanding credit assignment in long-horizon RL.

Entropy is the default signal agentic RL reaches for when deciding which step in a long trajectory mattered most. It’s cheap, it’s already computed, and it correlates with something. But correlating with outcome is not the same as locating the decision that caused the outcome — and on a real long-horizon agent, those turn out to be different things.

Read the full article on Substack: Where Does a Trajectory Actually Turn?