Do you think it's fair the those who are more educated or more skilled - like pigs in Animal Farm have more influence in decision making?

Sure it's fair.  In fact, it's almost impossible to avoid.

Think about how our society is.  The people in positions of power are just about all more educated than the average person.  They are all certainly skilled (at least in getting elected).  It would make very little sense to take a bunch of people who don't know anything and have them be in charge of making decisions for our society.

The problem with Animal Farm isn't that the pigs get to make the decisions.  The problem is that no one else has any input.  And the problem is that the pigs manipulate the facts to fool the people.  That stuff is way more dangerous to society than the fact that the people in power are more educated and skilled than the others.