Why don't you institute some type of algorithm similar to Google? You could deliver points just like you do now, on a one user one voat basis. But then applying a different algorithm to decide the visibility of a post. Where users with certain attributes voats count for more or less depending upon their user activity. For example maybe user's who have been around longer or contributed more content voat counts for more when deciding visibility.