> The Financial Times, for example, blocks any bot that tries to scrape its paywalled content, including bots from OpenAI, Anthropic, Perplexity, and the Internet Archive
But then it was not really open content anyway.
> When asked about The Guardian’s decision, Internet Archive founder Brewster Kahle said that “if publishers limit libraries, like the Internet Archive, then the public will have less access to the historical record.”
Well - we need something like wikipedia for news content. Perhaps not 100% wikipedia; instead, wikipedia to store the hard facts, with tons of verification; and a news editorial that focuses on free content but in a newspaper-style, e. g. with professional (or good) writers. I don't know how the model could work, but IF we could come up with this then newspapers who have gatewalls to information would become less relevant automatically. That way we win long-term, as the paid gatewalls aren't really part of the open web anyway.
Wikipedia relies on the institutional structure of journalism, with newsroom independence, journalistic standards, educational system and probably a ton of other dependencies.
Journalism as an institution is under attack because the traditional source of funding - reader subscriptions to papers - no longer works.
To replicate the Wikipedia model would need to replicate the structure of Journalism for it to be reliable. Where would the funding for that come from? It's a tough situation.
> Well - we need something like wikipedia for news content.
The Wikipedia folks had their own Wikinews project which is essentially on hold today because maintenance in a wiki format is just too hard for that kind of uber-ephemeral content. Instead, major news with true long-term relevance just get Wikipedia articles, and the ephemera are ignored.
> it was not really open content anyway
Practically no quality journalism is.
> we need something like wikipedia for news
Wikipedia editors aren’t flying into war zones.
Well, and it would be considered "original research" anyway which some admin would revert.
Original reporting is allowed and encouraged by the Wikimedia Foundation sister org Wikinews, which may be cited by Wikipedia.
https://en.wikinews.org/wiki/Wikinews:Original_reporting
Wikinews is on hold nowadays. Original research that is of real long-term relevance can go onto Wikijournal, which does peer review.
Statistically, at least a few of them live in war zones. And I'm sure some of them would fly in to collect data if you paid them for it.
> at least a few of them live in war zones
Which is a valuable perspective. But it's not a subsitute for a seasoned war journalist who can draw on global experience. (And relating that perspective to a particular home market.)
> I'm sure some of them would fly in to collect data if you paid them for it
Sure. That isn't "a news editorial that focuses on free content but in a newspaper-style, e. g. with professional (or good) writers."
One part of the population imagines journalists as writers. They're fine on free, ad-supported content. The other part understands that investigation is not only resource intensive, but also requires rare talent and courage. That part generally pays for its news.
Between the two, a Wikipedia-style journalistic resource is not entertaining enough for the former and not informative enough for the latter. (Importantly, compiling an encyclopedia is principally the work of research and writing. You can be a fine Wikipedia–or scientific journal or newspaper–editor without leaving your room.)
Those roles seem to be diverging:
- crowdsourced data, eg, photos of airplane crashes
- people who live in an area start vlogs
- independent correspondents travel there to interview, eg Ukraine or Israel
We see that our best war reporting comes from analyst groups who ingest that data from the “firehose” of social media. Sometimes at a few levels, eg, in Ukraine the best coverage is people who compare the work of multiple groups mapping social media reports of combat. You have on top of that punditry about what various movements mean for the war.
So we don’t have “journalist”:
- we have raw data (eg, photos)
- we have first hand accounts, self-reported
- we have interviewers (of a few kinds)
- we have analysts who compile the above into meaningful intelligence
- we have anchors and pundits who report on the above to tell us narratives
The fundamental change is that what used to be several roles within a new agency are now independent contractors online. But that was always the case in secret — eg, many interviewers were contracted talent. We’re just seeing the pieces explicitly and without centralized editorial control.
So I tend not to catastrophize as much, because this to me is what the internet always does:
- route information flows around censorship
- disintermediate consumers from producers when the middle layer provides a net negative
As always in business, evolve or die. And traditional media has the same problem you outline:
- not entertaining enough for the celebrity gossip crowd
- too slow and compromised by institutional biases for the analyst crowd, eg, compare WillyOAM coverage of Ukraine to NYT coverage
https://www.youtube.com/@willyOAM
> we need something like wikipedia for news content
Interesting idea. It could be something that archives first and releases at a later date, when the news aren't as much new
> a news editorial that focuses on free content but in a newspaper-style
Isn't that what state funded news outlets are?