So, the biggest potential cost saving solution seems to be the porting of the codebase so we can run on open platforms.This also aligns with Voat's pre freedom principals, and makes us more enticing to funding from similarly aligned groups like the free software foundation, potentially. While the git repo for voat is considerably outdated, there's little harm in poking around and familiarizing ourselves with the codebase. Other than donating and reaching out to potential donors, it seems logical that the best thing the community can do for the site right now is put our skills to work. I'm not the best programmer, I'm not a project leader, and I'm not someone with a ton of web development experience. What I CAN do is grunt work. I'm sure there's a number of users on the site in similar shoes that can cobble together a new stack for voat, even if it ends up needing a bit of outside help towards the end But we need someone to manage us. There have been a couple of comments pushing for such a community effort, but no where that has been explicitly marked as dedicated towards it. For initial planning purposes, I'm setting up this thread to accomplish that. Anyone interested, please post a list of technologies you are familiar with, level of experience, how much time you estimate you can put towards this project, and anything else you think might be relevant. I've also pinged a number of people who I know to be developers active on voat, or who showed interest in this in other places.
We especially need project leads, both someone willing to manage this clusterfuck, and people willing to lead particular subgroups when it comes time to figuring out who will actually work on what.
We need an organizational and code management structure. I suggest Git for code and trello for goals, with a slack channel for communication
We need people to verify code submisisons, both to look for potentially malicious code, but also to submit code to a second round of unit tests, and quash as many bugs as possible BEFORE we get too far past a particular area of the site's infrastructure, as well as ensure that everything is running as efficiently as we can make it. No use redoing the stack if it takes WAY more power to host the damn thing.
We need people who have the hardware and the knowhow to TEST our stack, once its in an alpha state, because unit tests aren't enough on their own
and finally we need people who can tap away at their keyboards, and produce functional code.
Fuck, there's probably a few roles i'm missing, but point is, Putt is doing ALL of this right now. He isn't going to work on porting when he's trying to keep us running, but the longer we wait, the harder this will be. So lets get started.
We also need to develop coding practices and standards. I think all of us can agree reading someone else's code that's inconsistently formatted and lightly commented is hell. We want to make any code we write as auditable as possible, for each other's sake, and the sake of the site's future if this project gets anywhere.
@VoatSearch @roznak @AnmanIndustries @JunOS @Ywis @ohnoitsaninja @Professor_de_la_Paz @1moar @TeranNotTerran @ShowMeYourKitties @bikergang_accountant @Caesarkid1 @derram
freeasinfreedom ago
Is it just the database or the whole site that is using microsoft features of Azure? Does anyone know how many instances of microsoft paid products they are running? Instead of a rewrite at the beginning how about figuring out what needs to run microsoft at this point and lets streamline that by figuring out how many licenses they would need on dedicated boxes. In most cases bare metal will outperform any cloud system for a lot less money and then we can have a donation drive for licenses and dedicated servers. Just moving off a cloud solution will save large amounts of cash if we can optimize what they are doing right now. If we could run the web server portion on Apache even if we have to proxy it to a microsoft backend that would be a start. Then maybe see what the traffic stats are and see about pushing the static content to some type of CDN to reduce load on the servers.
poo_22 ago
What was the motivation for rewriting reddit in the first place? I'm guessing it's political because Reddit is open sourced, is forking the reddit codebase and porting Voat's data to it not more viable? That to me seems like the quickest way to start hosting it on an open platform: it's the difference between migrating all of the data to a different schema and porting the entire codebase (as well as migrating data from SQL server to something open source).
IAmYourDad ago
I am a Java guy and I am interested in checking this out. I've done a few years of Enterprise stuff, HCM, etc. Never did any open source projects before. I figure why not take this opportunity to learn something and probably add the experience on the resume.
I can probably only commit about 1 to 2 hours a night though.
Donbuster ago
If you aren't already, please, join the slack team. Link is in the post.
Right now we seem to be leaning away from total rewrite and more for finding a system designed for making projects modular, for a minimum of actual effort. We get that set up, and test on the old builds of voat, send the solution to putt, get the site on a cheaper provider, then work on fixing the various problems around here.
IAmYourDad ago
Looks like you got everything figured out. I already register on slack using your link above. Is there anyway we can use fake names? Or do we have to use real names?
Anyways, I will be watching this thread for updates.
EDIT: Nevermind, I changed my name. Thanks.
trentlapinski ago
Former startup CEO who sold for an exit, was one of the first people to bring responsive web design to WordPress.
Currently a VP at a cloud computing and container management and automation startup.
I'm well versed in business development, project management, web development, system architecture, and devops. Although I am more of a tinkerer big picture kind of guy, and no longer code or consider myself a programmer.
My immediate thought process from what I know so far is: is either get the fuck off Azure, or contact Microsoft and see if they're willing to sponsor Voat for ad space.
Secondly, you should be able to virtualize Voat's current setup in a Linux container (Docker) without having to really touch much of the existing code (even though you're running .NET). https://github.com/dotnet/dotnet-docker-samples
Once you get the .NET code base running in a Docker container you aren't married to Azure. All you need now is a host that supports Docker, which is pretty much every major cloud provider. Once you're running in a Linux container you can pretty much host anywhere. I would also probably de-couple the SQL database and put it in its own container so you can scale the DB separately.
To save money on cloud hosting I highly recommend OVH, as they charge about 40-50% less than Google Cloud and AWS, and have cheaper bandwidth, memory, etc. Linode, and LiquidWeb are other options to take a look at. Or if anyone else knows of a cheaper data center with high availability in multiple geolocations?
This is maybe a week or two of work for 1 highly skilled devops engineer, maybe less, and will cut the hosting costs in half pretty much overnight.
Potential solutions to look at Rancher.com, mesosphere.com, https://kubernetes.io, or even https://traefik.io which you could install at the tech stack level and use as a basic devops management platform (although I'd seriously take a look at Rancher or Mesophere first).
Once that's done you can move Voat to any major cloud provider without having to rewrite the core code right now.
Phase 2: Core code rewrite.
Obviously it should be discussed, but an open source language with a larger developer community is recommended.
I would probably go with a NoSQL based solution as well for the DB, but also needs to be discussed.
What I can recommend from a web development standpoint is essentially start the rewrite by building out a core micro-service architecture and REST API. I would then rebuild the front-end using React, or another popular Javascript framework. Then you can continue to use a container based approach by hosting the core API in 1 container, DB in another, and front-end code in another then scale and load balance each appropriately based on need.
Take a look at http://www.telescopeapp.org - hell its open source we may even be able to borrow some code or ideas.
The major challenge with a rewrite of this style is migrating the existing DB into a new DB, but its doable.
As I said, I'm not the person to program this personally, but based on my experience this is the approach I would take. I would also open source the fuck out of all of this, and I also have a few monetization ideas.
Phase 3: You could allow people to launch their own Voat websites and sell them as a service at their own custom domains. Figure out what the hosting cost is, and double the margin and sell it as a SaaS. Then use the core API to integrate everyone into a single login / account management system. This would also allow you to scale individual communities separately, and put the burden of fund raising and payments on the community organizers. Obviously you will still have free subverses, but this approach would give the large ones the ability to launch their own domains, and cover their own costs, sell their own ads, etc. Meanwhile, Voat.co itself is merely an aggregator, and universal account management system, and SaaS provider.
RiverWind ago
Thanks for chiming in, /u/trentlapinski.
I liked where you characterised Voat as an Aggregator and Universal Account Management System. I also liked how you raised one possible means of distributing Voat by having individuals (Moderators, perhaps) host Subverses which they particularly supported. (Though I don't think that would be as strong a solution as using a blockchain.) I objected to most of what else you proposed, mainly on the grounds that it did not prioritize establishing and maintaining Liberty through the use of Free Software.
trentlapinski ago
Don't get me wrong, I think whatever we do should be open sourced. I completely support a free and open Internet.
I was merely suggesting a possible business solution that would create value for large communities, give them more freedom, while also allowing Voat to cover their hosting bills. That doesn't mean I think this should be the only solution for hosting a Voat community, it would merely be a solid service to be able to offer to the community to help offset costs, and create sustainable revenue. That doesn't mean the project can't be open source, and that others could go take the code and do what they want with it.
As for blockchain, not opposed to that as well but thats a whole other engineering challenge.
GenghisSean ago
I'm more on the ops side normally, but I'm very familiar with Python, C, Java. I know enough JS and ClojureScript to get by though. I can work on backend stuff in my spare time.
Donbuster ago
If you aren't already, please, join the slack team. Link is in the post
Cightline ago
Isn't Reddit written in Python?, why is Voat written in C#?
Donbuster ago
It is. Voat started as whoaverse, a hobby project of atkos. He didn't intend it to blow up like it did, he just wanted webdev pracitce. We've been building off the old code ever since.
Cightline ago
Maybe you guys could get a vote of the most popular language among users here. Then from that, rebuild it.
Donbuster ago
theres more to it than that, you need to consider suitability of the language. I'm seeing a few mentions of "R" in this thread. Obviously not worth building in, its a business language, not a web development language.
This was more of a recruitment thread. I posted an invite to a slack team, if you're interested, join it.
TremorAcePV ago
I noticed this comment in this other thread: https://voat.co/v/Linux/1867282/9146982
I have little experience in coding overall, but I am working on using PHP to script transmission and translation of data between databases (what I've used is MySQL and MS SQL, but honestly it's not much effort to switch to other types) currently and have had success.
I don't know if that'd be useful at all, considering that comment, but I figure since it would be a one time import, this is something simple I could do as a grunt worker to let others do other things.
Donbuster ago
Glad to have you onboard. Since this is going to end up being a volunteer project, we need all the help we can get
https://voatrewrite.slack.com/shared_invite/MTg0Mjg2MzQzNTg0LTE0OTUwNzc4NjMtZDcwZmNiODU5Yg
Secret555 ago
I own a website design and development company and sales have been shit this season. I would be more than happy to commit to this project. I would prefer to remain a drone and not take any management position if need be. Familiar with all the frontend scripting languages but also have experience with node.js, sails.js and other MVC frameworks. I am a LAMP developer and have worked on projects in the past to turn WAMP environment s over to Linux. Just tell me what to do for my community.
Donbuster ago
Good, voat doesn't appear to have many Node.js people.
https://voatrewrite.slack.com/shared_invite/MTg0Mjg2MzQzNTg0LTE0OTUwNzc4NjMtZDcwZmNiODU5Yg
Donbuster ago
Don't know why I didn't ping earlier, but @go1dfish @riverwind
And just to put this on your radar one more time: @puttitout
go1dfish ago
So my 0.02c on this project.....
Porting may be more trouble than it's worth depending on what you are trying to achieve.
How important is it that the existing data transfer to the new hotness?
It may be that a far easier approach is to start from scratch with cheapness in scale at mind.
Not saying this is absolutely the case, but may be something to consider if the task here ends up appearing too gargantuan.
Me personally, I'm taking a look at decentralization as a final solution.
Talked about this during the apaocalypse as well but the tech wasn't there yet, maybe with ethereum it is.
https://voat.co/v/announcements/1866053/9141606/10#9141606
RiverWind ago
Thanks for raising the issue of Data Migration, and also for your link to your excellent post on using the blockchain.
/u/go1dfish makes the valid point that running Voat on Ethereum would incur financial cost for users. Though it is not desirable to pay for a service, there are a couple of benefits that could arise:
1) Spam Reduction (paying to post)
2) ShitPost Reduction (a small deterrent might work wonders.)
3) Flair (You could acquire Prestige by making donations to others)
4) Voat Gold (By receiving Ether, a post could eventually be "Starred".)
I think go1dish is on the right track with decentralization. The best alternative to decentralization I can see at the moment is forking Reddit's code. Voat would then end up in the same hierearchical problems from which Reddit suffers.
Reddit_traitor ago
i suck at coding but i can help with testing and moral support.
Donbuster ago
Porting to a python based stack is certainly possible, after all, reddit runs on it. We need people from all areas, even if they aren't specifically programmers.
https://voatrewrite.slack.com/shared_invite/MTg0Mjg2MzQzNTg0LTE0OTUwNzc4NjMtZDcwZmNiODU5Yg
goatboy ago
This is what free speech looks like when it goes to war.
Donbuster ago
Good thing about open source- If we get this off the ground, it doesn't mean squat weather voat lives or dies. Free speech will live on. If I have to host it all of my personal boxes to keep a site like this open, choke my consumer internet, and make may virtualization sandbox machines worthless as a result, I will.
If it turns out voat's going tits up anyways, we could look into modifying for a distributed hosting platform
raw ago
I do science and I mostly use C, R, Java, Haskell and Julia. I am willing to spare 10-15 hours a week if you need programmers in the languages I mentioned. Please let me know.
Donbuster ago
As of right now, seems we are only likely to need the java (though I'd like to avoid java in general, even as one who works in it.) Do you by chance have any node.js experience?
regardless, join up if you haven't already
https://voatrewrite.slack.com/shared_invite/MTg0Mjg2MzQzNTg0LTE0OTUwNzc4NjMtZDcwZmNiODU5Yg
RotaryProphet ago
I've been doing major project platform ports for the past several years; HP-UX to Linux and Windows to Linux particularly. Most of these are very large software projects (1.5+ Million lines) written in C, which require significant work to bring to the new platform. I started as a dev, and have acted as a technical manager, and now am a project manager.
I've been lurking around here for a long time, but I'd be happy to help.
BurnedKirby ago
GPG Signed version of this post is here.
My gpg public key, and my keybase info.
My education in programming is about a Bachelors of Science in Computer Science, but I lack "real world" experience. The only reason I currently don't have a job yet is because I have had to deal with health issues, but I'm mostly recovered right now. I do have time to spend helping out.
I prefer C++ (not sure why myself but I do), but have programmed in Rust, Python, Java, Javascript, and C#.
Though I don't know C# that well as I've briefly used it for a team project for a class, but am willing to pick it up again.
I have been using Linux machines for personal use for maybe about 5-6 years now? My desktop and laptop run Arch Linux and am familiar with using qemu/virtualbox for testing.
I would not say I'm suited for a management role, but otherwise I think I can figure things out and write code. I just need to figure out C# again as working on a different language from what I'm used to can take a bit of time.
As an example of what I can do, I wrote a bomberman clone in a month for fun (very messy code base), and more recently made a BehaviorTree (for simple AI) that works with lua scripts.
jokersmild ago
I can provide an infinite string of beta testers.
I also want this to succeed so that I can use it for myself.
I had no clue that voat ran on something other than linux?
Disappointed ago
Runs on Azure I think and now the special startup company benefits they got are expired, the full cost is kicking in.
Disappointed ago
If a sticky asking for help on v/linux will help in any way just write it up and PM me so I can sticky it.
Edit: I put a sticky up in the meantime anyway.
PresnatDankusMaximus ago
Hi I'm willing to commit a few hours per week to this, I'm a software developer, I am decent with data manipulation and translating code, I'd love to help.
Donbuster ago
https://voatrewrite.slack.com/shared_invite/MTg0Mjg2MzQzNTg0LTE0OTUwNzc4NjMtZDcwZmNiODU5Yg
If you haven't already, here's our wip slack team
Zesty ago
Game Programmer here, but I have minimal experience with web frontend and backend. I could probably contribute a few hours a week. Please add me to the list of gruntworkers! I'd be so proud to have contributed to something as awesome as voat.
Donbuster ago
https://voatrewrite.slack.com/shared_invite/MTg0Mjg2MzQzNTg0LTE0OTUwNzc4NjMtZDcwZmNiODU5Yg
Hello grunt worker :)
goatboy ago
Could you make a very simple game and at the end the winners are redirected to voat. The boss character could be Anita Snarksleezian. After players rip her spine out, Street Fighter style, they are transported to /v/all and told: "Welcome Brave Einherjar to Valhalla. You will await Ragnarok here. The battle for Free Speech requires you to hone your skills in these subverses. Train hard and drink plenty of mead as you await the coming battle."
ShowMeYourKitties ago
It's been almost 5 years, but I have experience working with .NET (C#) MVC w/ Razor templates.
More recently I've done a lot of Python (Django, Flask) and can do all of the front-end too (js/css (preprocessors too)/html).
I've also got some devops skills that could potentially be put to use for infrastructure setup.
Donbuster ago
https://voatrewrite.slack.com/shared_invite/MTg0Mjg2MzQzNTg0LTE0OTUwNzc4NjMtZDcwZmNiODU5Yg
If you haven't already, join the slack team, and lets see what we can get done for planning
bikergang_accountant ago
Can I plug https://gitgud.io. It's run by the same guy that does catbox.
I'm going to try running the site on dotnet core and see what hangups come first.
We could go completely away from c#. This is how they get you on the db. You end up with two layers dependent on their environment and you can't simply write a replacement in one step.
I am shockingly ok with sticking with c#. I don't know it but I know you can't ask your main contributers to code in a language they don't know intuitively. I think core is the way to go even thought I could contribute the most on node.
Donbuster ago
Duly noted.
if you are up for it, would you join our slack team for more expedient communication?
https://voatrewrite.slack.com/shared_invite/MTg0Mjg2MzQzNTg0LTE0OTUwNzc4NjMtZDcwZmNiODU5Yg
bikergang_accountant ago
I'd really love to see a profile of the sql calls and their total execution time by template. Caching db is my jam.
@PuttItOut
For example this will produce a partner database with the same name +'.meta'. It stores data about the calls with a schema like this:
CREATE TABLE stat (template text,time int,calls int,past int, sig float);
time is total execution time.
calls is total calls.
past is how many times a different call shared execution time due to asynchronous.
sig is the standard deviation of the execution time (actually varience).
If we had data like that it's easy to identify one call to cache and that particular call speeds up 100x. I can even get past writes "invalidating the cache". I don't invalidate the cache. I write to the cached object and use a write to db when the key expires. Your most active data just lives in memory with no ipc or networking involved.
But the key is you need that profile information first. Your profiler needs write caching though so you aren't writing to a table for every read.
Donbuster ago
since I have been informed there's a ping limit of 3, and no one I pinged other than the first 3 responded @JunOS @Ywis @ohnoitsaninja
Donbuster ago
@Professor_de_la_Paz @1moar @TeranNotTerran
Donbuster ago
@ShowMeYourKitties @bikergang_accountant @Caesarkid1
Donbuster ago
@derram
derram ago
A bit outside my wheelhouse.
All I know is python -w-
Donbuster ago
You DID manage to write a bot before the API was out, though. If nothing else, you know the site's functionality damn well. We won't just need programmers, we'll need people to test the code, too.
VoatSearch ago
I'm in and support the primary importance of having good project leaders. Not my forte - I code and will happily do QA too
Donbuster ago
If you haven't already, please join slack
https://voatrewrite.slack.com/shared_invite/MTg0Mjg2MzQzNTg0LTE0OTUwNzc4NjMtZDcwZmNiODU5Yg
SwiftLion ago
I'm a lead engineer at a decent size company, working on a product that has 300,000 unique users a month. I've got a lot of experience working in an agile structure, as an engineer who's enjoyed doing it, and I've been on teams from 2 to about 9. I would have to sit down and really consider this impact on my time, but I believe in voat as a free speech platform, and I am eager to contribute meaningfully to this community that feeds my eyeballs their dank memes every day.
If you're interested in talking to me about my experience further, I could do a video call at some point. I have questions too about what size the community is, who has what skills to contribute, and some other details like that. But, I'm ready to consider offering project management and general management time.
Donbuster ago
https://voatrewrite.slack.com/shared_invite/MTg0Mjg2MzQzNTg0LTE0OTUwNzc4NjMtZDcwZmNiODU5Yg
WhiteRonin ago
Here read this: https://voat.co/v/whatever/1866574
AnmanIndustries ago
I'm no good here as I do not have the time to manage such projects. I already have to manage several actual businesses. I am always available for advice or guidance, but that would be useless in this venture as you need dedicated people who are familiar with what has been going on. I am still happy to assist indirectly anyway.
My applicable possible skills are in resource management, people management, project management, risk management, implementation and maintenance. Which is not very useful from an advisor stance. I am fluent in VBA, PHP and mysql, but so is like, every one else and I have no time for coding anyway. Im more of a business man and farmer these days. Anyone need help with a goat farm? (my wife wants goats)
With all of this being said and all of your attempts, putt has to figure this out for himself as he is the only direct access stakeholder. He needs to pick from the people he wants from the community in order not to get anyone who will leave backdoors open or sabotage things. I'm mostly a lurker so I do not have much evidence for myself anyway.
Donbuster ago
My point is that we work independently of putt, get a semi working version of the stack, then approach putt with most of the heavy lifting having been done already, rather than telling him "While you're busy trying to keep the site alive, here's more work." Sure, he still has to audit the codebase (methinks atko might have to come back in for a bit of work on that one), but this way he doesn't have to touch it until the point at which his involvement is absolutely essential
AnmanIndustries ago
There is a lot more to it than just make a new code. You need places to test it. Live data to test. All the other systems that run with the site. A place to keep all these tests. A place to manage these various server states. Then you need a means to manage it all. You need to screen all the people who have access to all of this data. The server accounts, the financial accounts (controlling expenses as a possible tax offset, being an account holder for bills), the business side (licensing, hosting, law).
You can build a new code, only to find out we cant use it and all that time has been wasted due to many possible outliers that are not considered without all other variables.
Donbuster ago
Of course. I have a small virtualization server laying about, and a few terrabytes of networked storage. I'm aware that integration and migration IS a pretty rough challenge, which is why such things won't happen if putt goes it alone, meanwhile hosting costs will continue to balloon. If we can get a functional stack for the site running on the test servers, we are then in a position to talk to putt and figure out what needs to be done to get the environment we build to be migration ready. Looking at other estimates around the web, this could SEVERELY decrease voats costs, but its too gargantuan a task for putt, especially when we're running on fumes financially. Plus, at the end, the site will be better for it. The site is very different than it was when atko laid down the first lines for whoaverse...
AnmanIndustries ago
I agree with you really. But with all of that, Putt still has to do the hard part without any "staff". That is, implementation, testing and changeover. If he organises this effort first, directly, then it will be way smooth.
Donbuster ago
I have already pm'd him and replied to a comment on the angel post informing him of this and asking for as many details as he can get to us. Weather he responds, or how long it takes him, is another matter
AnmanIndustries ago
Alright. Good luck with your efforts. Feel free to drag me in if you need another perspective. For now I can only really offer the occasional small cash investment.
Donbuster ago
if you want to keep an eye on us, feel free to join the slack team
https://voatrewrite.slack.com/shared_invite/MTg0Mjg2MzQzNTg0LTE0OTUwNzc4NjMtZDcwZmNiODU5Yg
Owlchemy ago
Not a coder at all, but applaud your effort, but been an internet guy from the good old days and have faked my way through much. I am a retired aerospace guy with time on my hands, so if there's anything I can do to pitch in, just holler. I could proofread, or do technical writing ... many years of tech writing and project management type stuff under my belt in aerospace electronics. Don't mind doing whatever, just not a coder per se, but generally a quick study. BTW, unless I'm mistaken, you can only ping 3 people per comment, so if I'm correct, many of those you pinged above didn't get it.
WeekendBaker ago
Project manager here. The most important thing is time, structure and follow up. I smell all 3 on you. Go for it!
VoatSearch ago
Hi mate. My thinking is that project management is the first thing. There are probably lots of coders around - I'm one and want to help - but I reckon managing the project, setting priorities, getting agreement on strategies, QA etc is going to be the main problem. A suitable subverse - is v/SavingVoat the best name? Not sure.
blkadder ago
I'd be willing to assist as I have a lot of the requisite background, however I say this with all due respect, as one or two others have pointed out, the challenges here aren't primarily technical. Voat needs a viable business model involving revenue, and absent that, it is hard to see a sustainable path forward.
Owlchemy ago
Sounds like a good enough sub name to me. BTW, will help as I said, but currently my wife is in the hospital and we're in the middle of a family thing, so have been just going on Voat to decompress here and there, so again, will help as I can, but may be a week or two until I have my full faculties back, so to speak. Good luck with your project, has Putt weighed in?
VoatSearch ago
Sorry to hear about your wife and family, I do hope everything works out ok soon.
Best recent info I can find from Putts is here: https://voat.co/v/announcements/1866053/9139508
So it doesn't sound like he wants to crowdsource development at the moment.
Owlchemy ago
Well keep me in mind if I can help ... not a coder, but have time and can do many other related things, if necessary. I want to see Voat succeed, which is why I post and try to be among the good guys. So anything I can do, I'm more than willing to give it a shot.
Donbuster ago
I'm fairly light on the experience side of things. Worked primarily in windows, next to no network programming experience, but willing to learn.
Python, Java, C/C++
Able to commit 10-15 hours a week to this