Ygthkb

There is a prince. (Or some other person of high rank.) And there is that prince's good friend, who, naturally, holds a somewhat lower rank.

There are two ways I could show the close relationship between the two:

1. 
   They can maintain the rank distinction. Horatio addresses Hamlet "good my lord" and "sweet prince". Sam addresses Frodo exclusively as "Master Frodo". Their close relationship is shown by other means. For example, Hamlet would not let Horatio speak ill of himself, and speaks to him openly about things that are close to his heart:

Hamlet: But what in faith make you from Wittenberg?
Horatio: A truant disposition, good my lord.
Hamlet: I would not have your enemy say so,

              Nor shall you do mine ear that violence

              To make it truster of your own report

              Against yourself. I know you are no truant.

              But what is your affair in Elsinore?

              We'll teach you to drink deep ere you depart.
Horatio: My lord, I came to see your father's funeral.
Hamlet: I prithee do not mock me, fellow student;

              I think it was my mother's wedding.
Horatio: My lord, it followed hard upon.
Hamlet: Thrift, thrift, Horatio. The funeral baked meats

              Did coldly furnish forth the marriage tables.

              Would I had met my dearest foe in heaven

              Ere I had ever that day, Horatio.

2. 
   They can throw rank distinction aside. This too is not unprecedented. In La Dame de Monsoreau, Chicot routinely addresses King Henri III by his first name, and even by endearments, like 'Henriquet'. He also uses 'tu' rather than 'vous' in addressing him (see T-V distinction). This is all based on reality, according to wikipedia.

In a fantasy setting (that is, a setting where I need not be bound by "what actually happened"), what are the advantages and disadvantages of each approach?

While @bruglesco makes a fair and worthy point, I'm in favour of showing degrees of formality.

Even in contemporary life, if one befriends a person one's age and a person 30 years older, one ends up using different ways of addressing those friends. In the past, the levels of formality would be even more pronounced not only because of age but because of class.

Personally, I find it jarring to read a historical piece which presents interpersonal relationships with the same ease as our time. However, I do advise to find middle ground.

I suggest determining four degrees of familiarity for the prince's circle, and I offer some suggestions:

1. very formal:

I bid you good day, my Lord Prince.

Sir Galahad, you may now take your seat.

Has the explicit purpose of showing where everyone stands in the social ladder, no warmth whatsoever

2. formal:

Good day, my Lord! And how was the council?

Take your seat, Sir Galahad, and do not bother me with that particular topic.

Use titles to show the appropriate respect for people's position, but the rest of the dialogue is more free.

3. informal:

Good day, Lord Henry! I hear the council was interesting!

Sit down, Galahad, and don't you dare mention that accursed council again.

Ha! Don't tell me I'm about to lose my head.

Better not. If I end up losing mine and doing something rash, I'd rather have someone with a working brain beside me.

Keep titles to the bare minimum and have the dialogue be much freer in order to make up for it. The prince will have the privilege to ignore titles of his close friends, but the nobles may or may not have permission to do so.

4. intimate:

Now that we're far from prying eyes and ears, do tell me what happened in the council, Henry.

Only the most intimate friends will be allowed to address the prince as if they were equals, although that illusion of equality may last only some moments.

If you can, lose the formality.

By definition, the familiar terms are, well, more familiar. Whereas formality is a constant reminder of rank. I for one felt the constant reference to Frodo as master by Samwise was jarring and anachronistic (to me at least.)

It won't be jarring with royalty, like it is for me with the hobbitses. It works for Horatio and Hamlet. But the most important thing to consider with that formality is that it is a constant reminder to all characters and the reader alike of rank.

So my questions to you are: Will the prince ever pull rank? If he is the type of person that is wont to do so then he will probably insist on the formality as a reminder. Will the friend follow a direct order unwaveringly? Even if it is against their character, moral compass, idealogical mindset, or best interest? If so they will almost implicitly call the prince by title as a sign of duty and respect. Or will the friend almost certainly follow their own path if they were forced to choose between the prince and their own personal guidelines? If so, I think they are not likely to use titles with their friends, unless held to do so by the sword (which would almost certainly end the friendship.)

I say it is ultimately a matter of the characters themselves letting their personalities show, however, I also suggested you choose one over the other unless you had to. The reason I said that is because formality will always be a small sliver of a wedge in any friendship. As a reader I will always be reminded that one of these people is more important than the other. And they both will too.

I think both ways of doing it are valid, and your examples already show that this is the case. Both can be interesting.

But you are asking about advantages/disadvantages. This is a much more interesting question than "which one should I do", so I'll try to live up to it. I will say though that I slightly prefer keeping the formalities because I think it's just another tool for world building and it would be wasteful not to use it. So my answer might be a bit biased.

Advantages of keeping formalities between friends

If done right, these formalities can be a glimpse into the fantasy world that we are inhabiting. For example, people from different cultures might have different ways of addressing one another. It can therefore be used as a tool for characterization, but also as something that can cause conflict. Imagine you have two close friends, and in a heated argument, the "lower" one stops addressing his friend as "my prince", but simply calls him "Henry". That might actually increase the tension. These social rules are exactly one of the reasons people like reading fantasy - they're a part of the world building. We do not want to have everything the way it is in our own world. (I would also say that if you do it, then think of your own formalities that you can use, and make them different for different cultures, and so on. If you write a fantasy novel but everyone talks like they did in medieval England, it's boring and very been-there-done-that.)

Disadvantages

After a while, it might grow tiresome for the reader, but I think that can be avoided with good writing. It should not really matter whether someone calls his friend "Jonah", "sir" or "my liege". The bigger problem might be that the reader mistakes the formalities that are very normal and common in this fantasy setting as something that is toxic within the friendship. You know, if you always maintain the hierarchy between two friends, it kind of seems like they are not friends at all! So you have to be very careful how you use it, because you do not want to give the wrong impression. You want to show that, on a friendship level, they are equals. If you have a friendship between a prince and a commoner, it might be necessary to show early on that the prince will literally risk his life or his good standing to save his commoner friend, or something like that, to make clear how true this friendship is for the two of them.

All in all, I'd say misrepresenting the friendship fundamentally is the biggest risk you are facing here, at least if you are good enough as a writer to keep the formalities from feeling boring or annoying.

Advantages of losing the formalities between friends

It's simpler for you and for the reader. It also quickly shows intimacy and triggers a positive reaction from the reader, because we tend to dislike upper class people who care about their titles. It might also give you more ways to play around with the dialogue, because if formalities are not a thing for these two, then it is also likely that nicknames, rude jokes and so on will come into play. It lightens up the dialogue considerably, basically.

Disadvantages

Readers might not feel drawn into the world, because there's a dissonance between this "modern" type of friendship and what they expect from a fantasy setting like the one you describe in the book. You see this very often with people who argue that "in such a setting, people would never do X" and so on. All the explanations for why this friendship is so special that they call each other by their first names etc. might ring hollow, like a lazy excuse for the author to fit modern sensibilities into a setting that wouldn't normally allow for them. You know, just like when characters in a medieval fantasy setting suddenly demand equal rights for men and women or something like this. You better be a really good world builder to explain why this happens in this world apparently a thousand years before it "should" happen.

Final thoughts

I think your second example of the guy who called his king by a nickname works because it is a true story. There are some stories that only work if we know that there is no author who specifically made the story the way he wanted to. If you want to have that kind of relationship, you cannot introduce it as an aside and then focus on your fantasy adventure - the story itself would have to focus heavily on how this special friendship came to be, and you would have to invest a lot of time to make it believable. Your example took the "shortcut" of being true. You cannot do that.

That said, I think your first example is also a good one that shows how weird it can seem at times to keep formalities in a friendship. I think we have all been there that we thought Samwise should just stop calling Frodo "Master". We do not want to read about a "friendship" that is actually a one-sided servitude without anyone talking about how unhealthy that is. (If I remember correctly, Frodo does comment on it, though, and does not especially like the way Sam treats him, so LotR has that covered.)

I guess in either case, no matter what you choose, you have to invest some time in justifying it to the reader. But I think the reader is technically on your side - we do want this type of friendship to work, but we also want a good explanation for it.

I think there is some measure of the characters and their history relevant to your question. I mean, If the prince's friend knows him from childhood, and they grew up together, they would naturally have an informal way of communicating. Think about Frozen: Elsa's sister only addresses her in a formal way when she is mad at her.

On the other hand, a close relationship may be romantic. They didn't grow together but the prince met the commoner girl in a tavern while he was out drinking, and got to know her better to the point where they fell in love after a few secret meetings. They will maintain a formal relationship while in the company of others (if they even happen to meet one another in a context different than a secret meeting). But they will not communicate formally in their secret meetings.

My point is, don't make this choice technical. make it about the characters. put yourself in their shoes and ask: "If I was the prince/commoner and this friend I have had known me for X and we share Y, how would we be talking to each other?"

I think there is a mixed use case. I think your Friend Frank can be smart enough to use formal addresses with Prince Peter if they might be overheard, and informal address when there seems no chance of that.

In any actual true friendship, I should think this must be the case, otherwise (speaking as a modern reader) I would feel there is an arms-length relationship where Frank is at all times subordinate to Peter, and that doesn't feel like a "friendship" to me, that feels like Frank is a pet of Prince Peter.

With mixed address, as a reader I can understand the necessity of the pretense in public; but we know privately that Peter considers Frank an equal in spirit, and Frank considers Peter a true friend, he doesn't feel like a servant to a master. I would make the pretense a secret between friends, Frank can even be wildly over the top in his addresses to the Prince in public, perhaps mocking the obsequious emissaries they have both heard in the royal chamber.

"Oh, prince of light and divine inspiration, prince of valor, prince of endless wisdom, prince --"

"Just get my damn beer, Frank!"

I think that imparts a little humility for Peter. This gives you chance, with Frank in private, for Peter to claim he knows his title, wealth and privilege are conferred by birth alone, that these were given to him, not earned by him. And though he will exploit that bucket of luck to the full extent he can, he knows his father too well to believe there is one drop of divinity within the King. And by extension, himself.

This makes the prince more relatable to readers, and the formality of address more of an insiders joke (which includes the reader), so to outward appearances it more closely reflects history, but nobody can claim your "enlightened prince" or friendship is impossible. We can't read dead medieval minds, and we can't trust what writing survived; it could all be lies and formalities.

Although these people did not have all the same facts and education as we do, they did have the same level of innate intelligence and reasoning as we do. So your enlightened prince may be an outlier, but isn't an impossibility.