What is the Point of Experience (Points)?

What is the point of experience points?  Nearly every game uses them, or at least some variation on them(Karma, Advancement Points, etc.), but what is the actual point of them? And more importantly, are they appropriate for every situation? and do we use them to do too much?

What is Experience?
So, at its base level, experience is a marker of character advancement, a way of representing character change through accomplishment.  As the character achieves goals, they get better at what they do.  But it is something more than that.  Experience points are not just a manifestation of learning, they are a manifestation of story form and to a certain degree a relic of the story forms of early games.  Dungeons and Dragons and fantasy narratives in general are based around a certain unlikely hero story form.  In that sort of story, which draws heavily on traditions of fairy tales, fantasy stories and bildunsromans, the hero begins the story the victim of a villainy or lack and cannot right that villainy or liquidate that lack until they are stronger.
Experience points become the method to explain that strengthening process.  It shows how Conan the boy can become Conan the Barbarian(and slay Thulsa Doom).  It explains how Meriadoc Brandybuck can go from scrumper of apples to slayer of Witch Kings(and liberator of the Shire).  It explains how Rand al'Thor can...do whatever it is he does(I didn't actually read that book series).   Experience points started as a structure to allow for that kind of storytelling.

Variations on Experience Points
In other games, Experience has been adapted but rarely abandoned entirely.  The classic model of Dungeons and Dragons is the linear advancement model.  Experience points buy levels, which progress along upwards power curves.  Other games use point buy systems, where you get generic experience points that are spent to raise specific skills/powers/abilities, representing a manifestation of effort at self-improvement.  This system allows for more nuanced development, but can also lead to oddly unbalanced characters or incongruous advancement(I learned a lot during our fight with that Mummy, like how to better influence local politics).  Some games use targetted achievement systems, where using a specific skill makes it improve.
Games also vary how experience is gained.  The classic model of Dungeons and Dragons is killing things.  Other games use story advancement, achieving plot points or good role playing to earn experience points.  Many targetted achievement systems mark experience through breakthrough moments, where outstanding success marks an opportunity for improvement.  My personal favorite is Dungeon World, where you get experience points whenever you fail at something.
What all these have in common is that they all represent an upwards power curve.  Characters start weak and get stronger.  But that strength is more than just symbolic representation.

Experience as Influence on Narrative
More experience equals more power, and more power equals more influence on the narrative.  Ultimately, a change in power level allows the PC to do more things, to interact with parts of the world that were inaccessible before, to take on enemies that previously eluded them.  This is both good and bad.  For games where everyone advances together and is part of a single team/faction, this can generally add up to the world becoming more the playground of the PCs.
However, this breaks down when you get to games that have multiple groups competing for influence over the narrative.  While experience points are not the sole method of influencing game narratives, they are a part of the overall equation.  When there is a significant power differential between characters in a game, those that have a lot more power are able to exercise a lot more control over the narrative. In these cases, experience point differentials can be a significant problem, because there is very little as disengaging as feeling like you have no influence over the game.

Arguing Against XP
This is one of the reasons that I'm strongly opposed to the current system of experience points being used in most ongoing LARPs and why I think LARPs can take something from standard tabletop practice.
1. I don't believe in XP as an OOC reward system
Some people have more time than others.  Some people have access to more money than others.  Some people have skills that others lack.  Depending on the game, these can be turned directly into narrative influence by volunteering/buying things for the game/making objects for the game.  This disengages people who don't have access to the same resources but still want to participate.
2. I don't believe in an arbitrary universal starting point.
In the D&D games I've played, when you bring in a new character, that character's level is related to the level of the other characters in game.  It might be average level-1 or same as the lowest level character or experience equal to whoever in the party has the least, but almost nobody forces you to bring a level 1 character into a level 12 game, even if everyone else started at level 1 initially.  But in many LARPs I've been to, new characters start out at starting level even if there are characters in the game walking around with hundreds of XP.  I think that LARPs can learn from tabletops here, establishing XP floors that allow new characters to feel like they aren't completely behind.
3. I believe in XP caps

There is a point of diminishing returns on XP as a narrative instrument.  I run bounded games, games with pre-defined beginning and ending points, which regardless of system, puts a cap on the total amount of XP that can be earned.  But not everybody does this.  I have participated in many games that had been running for years, with characters earning experience points the entire time.  In so many cases, I heard players talking about how they didn't know what to spend their experience on, so they were learning irrelevant skills or becoming experts on obscure lore.  This does more than just unbalance a game.  This cheapens the idea of character.  One of the reasons that characters have different skill sets is that it provides a method of focus sharing.  When a character has reached the point where they can solve all problems on their own without help from other characters, then the game becomes not a group participatory exercise, but an exercise in narrative masturbation.  XP caps limit that.  By having a maximum amount of XP, a character can only specialize in so many things and situations that require multiple degrees of expertise thus require multiple characters.  

One Solution

So, for the Changeling the Lost game that I am about to wrap up, I used an experimental system of character advancement.  Rather than have an experience point system that depended on attendance at game, all characters were given a total number of advancements(24) based on the total number of games(24).  These advancements could be used to advance any aspect of the character, so long as there was narrative justification.  

Players could spend them at any point in between games, starting at the beginning of the game with all of them spent or starting late in the game with none of them spent.  This put an overall cap on potential power and allowed for a variety of starting points.  It simplified calculation and character sheet tracking and removed arguments about power differential based on XP.  

Was it perfect?  No.  There is still a power differential based on system knowledge.  There were people who engaged with the system much more than others.  There was some confusion over the timing of things and honestly, when I could no longer print character sheets out for free, we went to a much more system light approach.  But I think it's better than the alternative, maintaining the good parts of XP while avoiding many of the pitfalls that I've seen.