The invented language of the Na'vi in Avatar raises the question of communicating with aliens, and for that matter how aliens might communicate among themselves. (The title, from a comment on a blog post at Language Log, was alas a mere typo, corrected to 'people who speak in Klingon or Na'vi.')
Communicating with aliens, by odd coincidence, also came up in comments to the last thread on Starship Troopers, if only backhandedly. Mutual understanding through dialogue, we are given to understand, is not an option with the Bugs. Heinlein does not bother to sneer at the idea of negotiating with them. When the MI capture a 'brain' bug, the Federation wants to study it, not interrogate it.
In Avatar (which, disclaimer, I have not seen) the Na'vi are perfectly capable of language - one that can canonically be written with the roman alphabet - but the corporate mercs are still not interested in talking. If the 16th century is anything to go by, this is all too plausible.
But supposing both parties want to talk, what then?
My uncharitable first reaction to written Na'vi, specifically the name Na'vi, is that Diana Wynne Jones has become our tour guide. Apostrophes in fantasy names are notorious. (Diacritical marks are exotic to English speakers, and the apostrophe is the only one we have on our keyboards.) But this is a quibble, because Na'vi is unabashedly put together as an exotic (to us) human language. Metaphor, in this case, trumps Hard SF literalism.
Real aliens might not make things so helpful, no matter how eager they are to talk. Broadly speaking there are two schools of thought about this. One is that aliens will be alien, their thought processes and means of expressing them so different from ours that real communication (not just a sort of black box interpretation of behavior) will be impossible. The other is that true, sentient intelligence can overcome these barriers, indeed in a sense is about overcoming these barriers.
The barriers - simple, profound, or both - could take varied forms. Only prejudice says that hive entities must be implacably hostile, but striking up a conversation with one could be difficult. If they have a brain caste of sentient individuals the problem is not unlike dealing with any alien race. But if the hive's intelligence is networked, it would surely be a very slow learner. (I believe a commenter made this point a few weeks ago.)
If sentient reasoning arises out of the interplay of the whole hive, a single simple thought might take form at roughly the pace that tidbits go viral among humans in a village or online, perhaps a couple of days for its consciousness to formulate the equivalent of a simple sentence. ('Hmm, this pattern looks non-random ...') Perhaps much longer. Any real dialogue with it might take generations.
But that is only a technical problem of sorts, and it has an established technical solution, the institution of the university, a tool for sustained inquiry that goes back nearly 2500 years. The real puzzler question here goes to a deeper level: How much of our reasoning, and theirs, is universal, and how much is bound up in biological experience?
There is a trope that mathematics, at least, is a universal language, but it depends on what you are trying to say with it. Music is richly mathematical, a transmission as unambiguously intelligent as the first thousand digits of pi, but what information does it encode? Writing about music, as the saying goes, is like dancing about architecture.
Art, in all its forms, is where Plato enters the primate house. It makes full use of our intelligence, perhaps the highest use. It is the marker of our humanity; the argument over whether Neanderthals were truly human, or not quite, is all about whether they produced some form of art, if simply applying makeup. But art is also all about apes hooting, jumping around, and being fascinated by bright colors.
Would it make any sense at all to non-apes? Would their equivalent, if they have one, make any sense to us? Could we recognize their art, even in a black box way, let alone appreciate it? Could any sentient race not have some analog to art?
If we encounter aliens, what do we try to talk about?
The image is of the Rosetta Stone.