Languages return one more entry then Linkedin UI shows

[fkrauthan]

Describe the issue
When looking on Linkedin I have two languages in the Languages section (German and English). But when downloading the JSON profile I get 3 languages (English is duplicated. I assume it has something todo with manual provided languages vs my profile language as one English was auto added as Native Speaker)

Help me reproduce your issue
Add one language (the same as your profile one with different level) and add an additional language. After that export the JSON.

Additional context
This is not very important for me personally just thought I report it.

[joshuatz]

@fkrauthan I was aware of this when I implemented #59, in 35cba20 - I probably should have documented it.

Unfortunately, I don't really know a "clean" way around this, not without explicit knowledge of private LinkedIn internals. Your guess as to the root of the issue is correct; there are two ways users can indicate languages - one is as the profile level, and the other, like you said, are the manual fields.

In the case that the user has manually provided languages, de-duplicating them across the default profile level languages is rather difficult:

• I wouldn't want to make the assumption I can just drop the profile level ones, as a user might not have remembered to manually add the same one to their languages section
• LinkedIn uses different hard-coded values for the profile section vs the manual languages fields. For example, if a user has their default locale set to English, the profile field would have a value of en, but the manual field might be English.
• While I have a feeling that the profile default locale field probably uses ISO 639-1 codes, the manual fields are less standardized. When I try to edit my profile, it lets me select things like Dutch, Middle (ca.1050-1350), which are not in ISO 639-1, but are in 639-2, but it also lets you save any string value to this field...

I think what I will do is:

• For now: add a little bit of code that tries to, at the very least, de-duplicate English / en entries, since these are common and I already know the strings to look for (EDIT: I actually already do this - I forgot I set this up in 35cba20).
• Future: Will leave this issue open, in case anyone has any extra insights they can share for a more full-featured approach.