For me, playing an RPG is about two things (having fun/being with friends needing no restatement):
- exploring a world/scenario by making choices for the character I have created
- feeling that those choices have led to progress in my character and the world being explored
But to know what these mean in game terms, we have to be specific about the definition of 'exploration'.
From an old-school perspective, that's easy. Exploration is literally physical exploration. Dungeons, caverns, sprawling necropolises, etc. Enter the dungeon, beat the monsters, find gold and treasure, sell the excess for better equipment, level up your monster killing abilities, repeat. That's the exploration-progress cycle in a nutshell.
But there are many people who play fantasy RPGs who do not focus entirely on dungeon delving. (Like me, and my groups.) Some people like politicking, some like family drama, some are big on war-stories, others enjoy gaining power through magic or fame.
The thing that all of these (including the regular delving) have in common? Your character learns something. They learn how to kill a basilisk, where the wizard keeps his lab, how to cast a spell, who murdered their uncle, etc. Exploration means nothing if a character isn't changed by it, and character change is universally driven by learning.
So for my tastes, the REAL cycle is: learn about the world or yourself, make a choice, repeat.
Enter 5e D&D. When I first took a look at those backgrounds, I thought "Thank god! A little something for the storytellers. Love it."
But now, I feel differently. It seems to me that these backgrounds are once again only a good way to increase storytelling and reward role-playing if you use them to learn more about the world and yourself. Otherwise, they are just another few pluses and minuses on your character sheet. In fact, they are almost worse than just a few pluses and minuses, because it implies a kind of en media res about the whole thing.
You were a soldier. All the development and learning from those days is said and done. Now you're a treasure hunter. Yeah, you might use that military past to help you become a better treasure hunter, but your struggles and experiences as someone who was in the military are done. All of that exploration and learning was done before the game, and now cannot be done again. The story is half over before the first session of the game....
This even goes beyond backgrounds. Consider the wizard class in any edition of D&D you desire. It is always implied that the wizard isn't just some guy who picked up a spell book yesterday. The fighter and the rogue can just be "some guys" before session one. But the wizard always has a backstory that could be the premise to several young-adult fantasy novels.
D&D isn't designed to tell the story of a normal guy becoming a great wizard. D&D is designed to tell the story of an okay wizard becoming a great wizard. There is no inherent problem with that, but it will frustrate you if that first kind of story is the one you want to tell.
Or, you could just play the sorcerer. In my years playing 3e, I loved the sorcerer class far more than it deserved. Mechanically, it was near identical to the wizard (just much worse), so it isn't very inspiring on that front. But the sorcerer is arguably one of the best storytelling setups in 3e. The main points were roughly this:
- Nearly all sorcerers start gaining their weird powers around puberty.
- They are shunned and kicked out of their homes/villages/cities because of their weirdness.
- Therefore, the "average starting age table" in the 3e book made sorcerers the youngest. Seriously, it was like 13 + 2d4 years. Like 15-21 years old.
- And lastly, most sorcerers, since even their parents shun them, clearly have no idea where their powers came from. Unsurprisingly, most are motivated to find out.
That setup is all forward momentum. The backstory is perfect, because it is a mystery. It is something that creates a goal for the character, rather than just giving bonuses or letting your character say "oh yeah, I know that" during some random GM monologues about the lore of the world.
The sorcerer agrees with my tastes even further, however, because it most clearly follows the "exploration = learn something" rule. By delving deeper into the origin of your powers, you will inevitably learn more about them and become more powerful. The story and mechanical goals of the game intertwine.
For my money, I'd love it if every character had as much built-in interest in their immediate future as the sorcerer class in 3e. I feel 5e backgrounds got close, but where there should have been more "and here is what an ex-soldier might be looking to achieve in the future..." there were bonds and flaws and personality traits instead.
Those help define how your character might do something, but they don't help you with what your character wants to do.