Genuine question: how do you scale content moderation to millions of posts without hiring millions of workers at astronomical cost (who can make mistakes, like we’ve seen with DMCA take down notices, YouTube censorship when not justified, AppStore censorship,even GitHub made these mistakes, etc... some of these mistakes ruin peoples’ livelihoods).
Is there a “right way” to do this that protects everyone?
If a business can't run profitably and legally at scale, then that business model is a bad fit for that scale and shouldn't exist at that scale (presuming the laws in question are reasonable, which I think 'banning child porn' qualifies as.)
It's super weird that people seem to constantly start from the premise of, "But their business couldn't possibly [comply with law] profitably at scale! Oh well. I guess until someone comes up with a feasible solution, we'll just have to exempt them from being responsible!"
Imagine if pharma companies went "gosh, but hiring quality inspectors to make sure all of our drugs are untainted just doesn't scale!" I have trouble imagining you saying, "Well, they obviously can't sacrifice scale, so let's just wait and see if a better quality inspection solution comes up."
> Imagine if pharma companies went "gosh, but hiring quality inspectors to make sure all of our drugs are untainted just doesn't scale!" I have trouble imagining you saying, "Well, they obviously can't sacrifice scale, so let's just wait and see if a better quality inspection solution comes up."
Every drug has acceptable levels of impurities. There is a non-zero acceptable level of rat feces in your food. Your bank rounds the value of your account to some decimal place. The various parts of your car have some mean time between failures.
In the real world we place bounds on how much bad stuff is acceptable, and these bounds are determined mostly by practicality and economics. "Ban all childporn" is certainly a reasonable philosophy, but it is no better a law than "ban all impurities" which would shut down every drug manufacturer in the world. A real law takes the form "perform these steps or reasonable equivalents to limit X to a tolerable level." You can claim that reddit is not taking a step it is required to by law, but it's not breaking the law just because it hasn't developed a perfect solution to a problem that likely can only be solved approximately.
The only way this comparison to drug companies makes sense is if the drug companies had community sourced supply lines and were selling drugs a third party contributor had made in their bathtub alongside their normal drugs.
> "Ban all childporn" is certainly a reasonable philosophy, but it is no better a law than "ban all impurities" which would shut down every drug manufacturer in the world.
This would be a reasonable argument for a company like YouTube which has spent tens (hundreds?) of millions of dollars putting controls in place. If PornHub had iron tight controls and one of the actors was caught using a fake ID, then it might be reasonable. But they didn't have iron tight controls... or apparently any controls. There was zero verification that participants had signed releases or were of age.
PornHub's controls were complete shit.
Maybe some people want drugs from Pfizer's "Community Sourced" drug program... I sure don't.
This is an extremely weak argument. At a certain scale, policy becomes about statistics rather than absolutes. This applies to absolutely everything, including the laws themselves.
Companies in all industries have to occasionally issue recalls on dangerous or faulty products, just like PornHub or Reddit have to occasionally delete stolen or illegal content.
Terrible example. What if that was true? You think that we would just shut down the companies that make essential drugs rather than change the regulation?
People are going to watch pirated porn. If they can't find it on American sites they'll go to sites that aren't effected by American laws/regulations/journalism. That would probably make trafficking worse as the new site has no reason to take down any videos as their value add is refusal to censor.
What you're effectively saying - just like all arguments of this form, which seems to be quite popular lately - is that ordinary people should not be able to share their own content that they create online, because it's too hard to 100% police everything that they do. (Presumably including sending things directly and privately to people that they know, given how much better a channel that is for child porn than somewhere like Pornhub.) That the internet should be a consume-only place for the output of big corporate content factories whose content can be verified. Except framed in such a way as to make this seem like an anti-corporate stance, rather than the massive pro-corporate grab it is.
You do realize the success of the platforms is the result of finding legal loopholes.
That's the innovation. What if we are a media publisher but than without worker rights and unions? Youtube.
What if we start a hotel franchise but without any regulation or oversight or liability? Airbnb.
What if we start a cab company but make all our employees independent contractors, ignore all regulations and put all liability and legal risk on the independent contractors?
Silicon Valley is nothing but a bunch of young spoiled boys asking themselves the most American business question of all: have you tried slavery yet?
Business model innovations where you make money while someone pays all the costs.
>Is there a “right way” to do this that protects everyone
There is no investment if you do that, because the competitive advantage comes from the stealing. Nobody is investing in technology. Nobody is doing actual R&D. It's all about the viability of the scam. How profitable is this loophole?
We go out of our way to engineer protocols and technology and platforms that make it impossible to do that the right thing (by design). From p2p networks in the 90ties to crypto currencies. From YouTube to Spotify.
It's a dog eat dog world and I'm not here judging any individual. But let's not be naive.
Theft is the plan. Abuse is the strategy. Fortunes are not built on sweat. The people who work hard are the ones you hire to clean our house. The ones that pick our vegetables.
But we at least show the respect to others to not lie about the true nature of the successes of our industry.
>> We go out of our way to engineer protocols and technology and platforms that make it impossible to do that the right thing (by design). From p2p networks in the 90ties to crypto currencies. From YouTube to Spotify.
I don't understand this bit. So are you saying crypto currencies are making it impossible to do the right thing?
>So are you saying crypto currencies are making it impossible to do the right thing?
Cryptocurrencies are designed to subvert regulation and oversight. That's the selling point. That current societal infrastructure to protect innocent (sometimes gullible) people from everything from weapons trade or pyramid schemes.
If you want to 'make it do the right thing', thats already exists and is your wallet. Thats called money. A ledger with a tax identity. A bank account that requires a legal id. There is no point in burning all this electricity to verify those hashes, except to circumvent regulation and law.
I can't start a pyramid scheme and steal money from gullable idiots. Thankfully, an ICO is completely legal (for now). If you want to sell bread to hungry people, use money. If you want to sell guns to a hitman, use a cryptocurrency. Thats why cryptocurrencies are economically viable compared to money. Because they make it harder to apply the laws we voted on democratically, like how you can't sell children for sex.
The reason any money out there is investing into cryptocurrency technology, is because of the expectation of moral subserversion of society. Its a not a side effect. Its the bussiness model.
What do you think Venture Capital considers some startups worth so much money? Because they see the potential of price dumping and becoming a monopolist. If markets were actually properly regulated to always keep competition alive, people wouldn't be investing millions into apps and websites, they would be investing in actual R&D and hard science.
But doing the real work, can never compete, with just plain cheating. You always make more money being the bad guy. Even if a lot of startups have sincere intentions (Facebooks' mission was to connect people, not destroy democracy), the money will chase the decision down.
I think this is a point that tech folks like to sweep under the rug all too often.
All of these sound wrong:
- Walmarts are huge! It's impossible to build fire suppression at that scale.
- Ford builds an insane number of cars every year, it's not possible to ensure each car is safe at that scale.
- The United States burns 20,000 barrels of oil PER DAY. It's simply not possible to ensure a "safe" product at that scale.
- This buffet serves 20,000 plates per day, it's impossible to ensure food safety at that scale.
Each of those problems has been ~solved. Or those business wouldn't be operating. You only need to moderate a million messages if your customers are posting that many. If you can't afford to hire moderators to handle those million messages maybe it's your business model that's the problem, not moderation.
Fire suppression systems fail, cars get recalled or malfunction, people get food poisoning in restaurants or from food bought at a supermarket. Any product, not just tech, runs into problems when manufactured at scale. Tech is more visible because it is newer and in the spotlight. Nobody says "stop manufacturing cars" when one manufacturer issues a safety recall because cars have been around so long. The entire premise of producing 100X fewer cars so we can hand-craft cars and hand-check them is ridiculous - we simply accept the occasional manufacturing defect or product recall as the rates at which they happen are so small. These are all rounding errors, and the benefits of scale far outweigh the cons.
The same applies to tech. Content moderation has been ~solved as well. The overwhelming majority of content on social media platforms, websites, etc is regular, legal content. The illegal content that makes it through the filters and stays up for a significant period of time is a rounding error.
> I strongly disagree. Pornhub just purged 10 million videos because they couldn't moderate out child porn and rape videos.
PornHub is not a good example here. Reactive content moderation doesn't work for porn, because it's too hard to distinguish a positive from a negative. Content moderation for porn has probably been solved...by using a per-clearance model coupled with strong identity verification.
> There's terrible content all over Facebook
That's probably true, but the worst appears to be much harder to find. It's a lot easier to detect and ban all porn (especially if you can ban all nudity) than just certain heinous kinds of porn specifically.
Facebook seems to have more problems with trickier things, like hate speech, holocaust denial, and mis/disinformation. We have pretty good technology now for recognizing things in images, but we don't nearly technology that good for understanding the meaning of speech and text.
This is the error. The problem isn't scale. If YouTube has 500 hours of video uploaded per minute and some other site has only one minute of video uploaded per minute, the smaller site is no better able to handle it, because they don't have proportionally any more staff than YouTube.
The actual problem is marginal cost.
If you have a Walmart store, it probably costs over a million dollars to construct it, and takes in several million dollars a year in revenue. A $10,000 fire suppression system may be expensive, but it's manageable. And if there are a thousand Walmart stores all over the country that each need a $10,000 fire suppression system, that scales just fine. You have a choice between an unsafe Walmart and a safe Walmart whose proprietors make slightly less profit.
The marginal profit from hosting a given video is pennies. That means any solution that costs more than pennies isn't a choice between unsafe and safe video hosting, it's a choice between having video hosting and not having it. Scale has nothing to do with it; greater or lesser scale doesn't save you from a cost that exceeds your entire margin.
> Is there a “right way” to do this that protects everyone?
You do it the way the porn industry has for years. You get consenting actors. You get releases from the participants. You keep track of who is in the videos.
If you are hosting third party content, you verify they do the above.
This isn't like hosting a Disney movie on YouTube. If you screw this up, there are real victims here. There should be some weight and repercussions to posting child porn, revenge porn, or any kind of porn where some of the actors have not signed off on it.
Not saying it is a "right" way, but this video of the ActivityPub conference contrasts content moderation of the big centralized platforms to how moderation is conducted on the decentralized fediverse (e.g. Mastodon, Pleroma, PeerTube, etc).
Spoiler: where the latter makes it way more manageable, and for the former it is near undoable.
Is there a “right way” to do this that protects everyone?