Academics have a duty to defend open knowledge systems like Wikipedia
blogs.lse.ac.uk148 points by jruohonen 8 hours ago
148 points by jruohonen 8 hours ago
> Just as academics must cite any AI-generated machine outputs they rely upon in their research and writing, they can also cite Wikipedia if they use it in their work. Doing so can highlight Wikipedia’s value to other academics as a source of knowledge and provide greater transparency in one’s research processes. Despite longstanding skepticism of Wikipedia among some scholars, citations to Wikipedia in academic journals appear to be increasing – even in high-impact publications in subject areas like the health sciences.
As someone who's contributed to Wikipedia and seen how the sausage is made: that's stupid. Even Wikipedia itself considers wikis (including itself) to lack sufficient authority to be cited as a source.
Such citations could be easily gamed (if you cite a particular change) or are useless (the content you cite may not be there in the future), and are obvious pathways to "citogenesis."
Academics citing any website should consider also pointing towards (or at least creating) an archived version via something like the Wayback Machine. Wikipedia is not the only website that changes over time, and most don't have a easily accessible revision history.
> Academics citing any website should consider also pointing towards (or at least creating) an archived version via something like the Wayback Machine.
It's probably better for the user to search the original URL themselves on the Wayback Machine. If you cite with a Wayback Machine link when the original link still works, you're just creating needless work for a non-profit. An archive isn't a CDN.
> or at least creating
I agree on creating.
Or at least have the decency to make sure there's an archive of it when you looked at it - so many times I've found sources sited and Wayback as one from two years before the cite, and one year after; completely missing what the cite was about.
The wayback machine doesn't archive every version of every page. If it's important/authoritative enough to be cited in an academic work, it's important enough to be archived with it. Universities should be investing in that kind of infrastructure, and it's shameful that they're not already experts.
> Universities should be investing in that kind of infrastructure, and it's shameful that they're not already experts.
Do you also expect universities to have an archived version of every article and book their professors cite?
This is not a honest comparison. Books are mass produced and don’t rely solely on their authors paying hosting bills and willingness to not change stuff to keep existing.
Relying entirely on a single nonprofit (archive.org) to provide a honest snapshot of every cited link, at the time of citation, sounds much more unreasonable.
A lot of national libraries also undertake web archiving within their scope (e.g. under their national TLD) and I've sometimes found that they have items not covered by Wayback, but it's not as easy to request additions to it. The risk of having a single point of failure is certainly real, and it would certainly be nice for a coalition of libraries to maybe fund a second alternative and mandate the use of both, or something like that.
Yes. 100% yes. Obviously and clearly, yes. Universities have huge libraries and archives. It used to be a point of pride how big their libraries and archives were. University libraries bound and collated their periodicals. They converted them to microfiche to save space, but the integrity was paramount.
Yes. Libraries should be able to back-reference every citation that is included in every paper that they publish, and so should the journals. That is paramount to the scientific method. Anything that you have not verified yourself should have a clear citation that you're able to prove.
For static content, the best option is probably to download the webpage as a PDF and add it to the appendix of your report.
If you look up something on Wikipedia and include it in your work without citing it, it's an even worse pathway to citogenesis, since someone trying to check whether you can be used as a source for Wikipedia won't know that that's where you got it from in the first place.
Of course for most things you should have no need to refer to Wikipedia at all, and hence also no need to cite it.
Recently I did a deep dive on some minor historical event in the 1800s that I won't bother to go into detail here, except to say that it has been claimed by an activist editor who has made it a personal mission to re-frame the event as an example of "westerners" committing sexual violence against helpless "easterners".
While certainly this kind of story is common throughout history, in this case, there's absolutely no documentary evidence of the event, and what little evidence exists points in the other direction. This is not just the weight of evidence -- it's what all current historical experts believe. It's what all major encyclopedias say about the event. Nonetheless, the Wikipedia editor clings to a bad reference (in an extremely biased book) that makes an off-hand assertion. This assertion, in turn, is sourced with only a single citation to a 1960s-era historical novel that a) didn't actually support what the author claims, and b) essentially made up signficant details of the event in question, based on popular movies and rumors of the time. It's garbage all the way down.
Nevertheless, anyone who edits this article to reflect accurate current knowledge is immediately embroiled in a war with the editor, because unfortunately, there's no "trusted source" that says the opposite of the bad citation. And so the lie remains. To fight this battle would require escalating the issue and possibly getting the editor removed. Who has time?
Again, this is a minor historical event that nobody cares about, and you still cannot trust it as a source. I have seen enough things like this across Wikipedia that I no longer trust it for anything that might intersect with modern political opinions.
More generally, if there are two incompatible versions of an article to be had and editors can't write a neutral article (could have various reasons) initially the largest group "wins" which may be overruled by so called established editors which at times works out even worse. If the topic is actively propagandized or insulting to one or more people the thing may drag out for years.
Yes, exactly. I think it should be obvious [1] to anyone that Wikipedia is unreliable for extremely contentious subjects or modern politics. But if you point to those as examples of editorial bias, you get mired in of-the-moment political bickering. The reason I raise this example is to show that even in a boring area that should be unrelated to modern politics, you still see the rot.
[1] "should be" is doing a lot of heavy lifting in this sentence. Unfortunately, too many people have had their higher reasoning centers liquified by partisanship, and cannot acknowledge inconvenient facts even when they're blatant.
Are other encyclopedias or sources of information less or more likely to fall prey to this kind of bias?
Maybe in Wikipedia it's just more open, so you notice it more?
I've also noticed bias, especially in anything political (or that admits different political or ideological interpretations, such as history). However, I think you have one thing backwards: the more obscure and niche a topic is, the more likely you'll find it biased, since it has fewer editors looking at it.
Encyclopedias in general are not primary sources, and should never be cited.
Just say which article this is about. You're leaving out a lead someone else could use to kick out that editor, or fix his mistakes down the line when he's gone.
I don’t want to get accused of brigading, and also, I think it has the potential to be a distraction from my point.
I was able to edit this to fix it, but the geography of New Zealand page had a reference to Southland having a "tundra like climate." Invercargill, which is in Southland, actually has a temperate oceanic climate and a record low of -9C or 15F.
I find wikipedia is really vulnerable to Gell-Mann Amnesia where any article about a topic on which you are very familiar has blatantly incorrect information, but then you open a different one and read it as though it were true. Of course this is true of other sources, famously newspapers, but I think wikipedia is more vulnerable, especially on topics that are very emotionally salient to its power editors.
If you had to get in a fight to edit that, it really is the height of wikipedianism.
I can totally see how this kind of question would intersect with climate activism though.
I didn't really have to fight to edit it, but it had been in the article for many years, despite being blatantly false (tundras have mean temperatures below 0 C for 6+ months out of the year, while coastal Southland has a mean temperature of 5 C in July, the coldest month). My best guess is some kid thought that Southland was cold and knew that tundras were cold, and added a sentence to the article comparing Southland to a tundra.
Whenever I catch something like this, I wonder how many things like this are in there that I don't notice. And this one isn't even really touching anything that is controversial.
Exactly. They also game the “no primary sources” rule to write pure fiction in modern political articles because a “trusted source” (aka a journalistic rag they agree with) purposefully didn’t include that information, and in fact may have blatantly falsified events.
> Such citations could be easily gamed (if you cite a particular change)
What do you mean by "gamed"?
If the source doesn't agree with my world view you may not use it.
My favorite version is when topics are stuck decades ago and newer sources are rejected as an attempt to make the topic look real
> Even Wikipedia itself considers wikis (including itself) to lack sufficient authority to be cited as a source.
Those two things are not alike. Nobody can view themselves as sufficient authority to be cited as their own source.
(The late Victor Mair was an exception; he was happy to "support" his own claims by citing wikipedia articles that cited unsourced claims from him.)
Yes I was always taught in school that one shouldn’t cite encyclopedias. Encyclopedias, including Wikipedia, are tertiary sources. When writing papers in school and college, secondary sources were what we were supposed to cite.
> they can also cite Wikipedia
Academics require primary source evidence - e.g., experimental or documentary. Wikipedia is a tertiary source; by policy it forbids primary sources and requires secondary sources.
Problems with wikipedia:
* one source of truth. Not useful for controversial topics
* On topics like graph databases (where I've spent 10+ years), the content doesn't reflect best practices (e.g. VLDB papers)
* Someone with sufficient resources can remove content in the name of "clean up"
* Google <-> wikipedia effect that others discuss in the thread.
Indeed, broadening what academic scores mean (past a citation number) and letting them be recongnized by contributing to open knowledge like wikipedia (vs closed private journals) is the counterbalance we need.
> The fulfilment of ethical obligations to support knowledge communities, however, requires support from institutions of higher education. Currently, most academics are evaluated on their scholarly and research achievements, teaching performance, and service duties to their departments. Public service, like contributing to Wikipedia, goes largely unrecognised, if it is even counted at all. If academics are to fulfil their responsibilities to protect knowledge and knowledge communities, institutions must recognise efforts like contributing to Wikipedia and provide resources and support. Doing so may also help to address the public’s eroding trust in higher education itself, as a greater commitment to serving the public would reassert the role of universities as stewards of knowledge that exist to serve humanity – a role shared by Wikipedia.
I’m old enough to remember when Wikipedia was not allowed to be cited in high school papers because it wasn’t a sufficiently authoritative source. How the turn tables…
That time was also when people could just add useful information to articles without some self-declared gatekeeper/“curator” removing everything they cannot grasp or simply deem uninteresting again.
Wikipedia is being lost to ideological bias.
That is a less useful observation than you might think; Wikipedia is fundamentally an ideological project. Obviously there is a bias.
The question is whether it is a bias to honest citation of sources that are expected to be factual. As long as they maintain that culture they are doing the best they can.
The funny to me is that without context there’s no way to know which ideological bias you mean. Just looking at the parts of Wikipedia I routinely use I don’t see much of a bias beyond one that favors more verbose articles.
I’ve routinely heard “Wikipedia is becoming ideological,” and it’s usually action flip as to whether they thinks it’s too far to the left or right.
What academics need to defend is wikipedia in latent space. Not walls of text that reflect the opinions of a few who have the time to edit that the rich and famous can optimize.
That is why we are building latentpedia.org.
What on earth did we do before WP? I'm old enough to remember the olden days before WP existed and the practice of reading the news.
I also remember the whispers and chatter outside of and around the 'papers. Those whispers etc would reinforce a later article.
What is happening now with news ie top down dissemination of world events is not new but what is different today is the immediacy of commentary by the hoi poloi - you and me and who knows what from across the world ... and bots.
I think we have to get to grips with the fact that an event happening within one corner of the world measuring an inch squared can rock up on the agenda of anyone, anywhere across the entire world.
I miss making monthly trips to the local university library as a child. I would have some infatuation, often related to electronics or programming, and would get a big book on it and just dive in for a few weeks. There were magazines on microfilm. It felt like a privilege to get that information. You had to wait to get to it and would only possess it for a while. The ideas would simmer for long stretches without any ability to instantly cross-check your information or get clarifications or feedback on work. Trite, I know, but I feel joy just thinking about that whole experience.
> What on earth did we do before WP? I'm old enough to remember the olden days before WP existed and the practice of reading the news.
Libraries and encyclopedias were my mainstays until Wikipedia came along. Many libraries still have a current affairs section that covers the news.
As someone who is old enough to remember the start of Wikipedia. Academia originally had issue with wikipedia due to the lack of academic oversight and therefore didn't allow it in citation for academic work. Now they have to defend it?
Editors are still not allowed to have credentials. The absurd argument is that credentials can never be validated.
I don't know how scientific journals manage to exist but technically it could be as simple as a link to the user page on the university professors profile page.
Defense of open knowledge will appear when they attack the monopoly of Elsevier and other publishers through innovative systems for peer-evaluated papers, not by trying to resuscitate the body of the spook-filled Wikipedia.
Great, then AI companies should divert 5% of revenue to sustain Wikipedia and researchers...
Defend Open Knowledge...
Yet at the same time their CloudFront WAF decided to block IP coming from Singapore ISP...
So much on defending open.
I’m pretty sure wikipedia has a bunch of mirrors, including one in the anti-social network nostr.
Downvote away, but: Academics have a duty to explain their years of inaction while their students have wilted and faltered as a direct result of academics not understanding how to to be effective in the real world in which their students are expected to be productive members. Those that cant, teach, and teach ridiculous fallacies they personally subscribe to. These people are too weak to stand up to their own bosses and boards. Are those the meek and weak we want teaching? Instead of real critical thinking they instilled a twisted sick worldview about social issues that have no connection to creating market value and being a good, supportive human that lifts all boats. They focus on things no businessperson would even consider engaging while missing the forest for the trees. Wikipedia is the last concern of this rapidly imploding rightthink crumbling empire, on its last legs from those whose families bore the ultimate cost to get their kid a degree that is increasingly worthless.
I propose our leadership cut funding and fine any program in the US where a professor is not actively working in their field at a minimum part-time. These people are so out of touch it is literally sickening when you think of all the negative downstream effects. Citizens should demand better of their tax dollars - academia in its form today is a cancer that is holding back your youth from greatness and robbing our nation of the qualities that made it great, kneecapping our young talent for independent thought before they even figure out who they are.
They're academics. The job is to say things that are true inosfar as a community of experts driven by rigorous argument believes.
Whether or not that has any effect on the real world is not their concern. The academy is proud of their record of interest in things that have no applications whatsoever in the real world and ideas that turn out to be extremely harmful if applied. The people pretending that academia is some sort of vocational system were wrong and are to blame for the academics doing exactly what they were expected to do.
Although yes to the idea that taxpayers shouldn't have to fund running everyone through university. The academics are often opposed to the taxpayers, it isn't fair to make the taxpayers fund them.
Open knowledge systems like Wikipedia are not really defensible. Letting anybody edit any page is a great way to get AI and human slop everywhere. It makes it extremely easy to insert knowledge with dubious quality and sources.
We need "open access" knowledge systems with known authors who are sponsored by non-profit and educational organizations and its their duty to defend their reputation.
Humanity needs more openly accessible curated and factually checked information and "open to modify" systems are bad at in all.
I used to edit Wikipedia as a hobby with the goal of getting as many incorrect facts in it as possible, and seeing how long they lasted before being corrected. These were really minor things that hardly anyone would care about so they would often fly under the radar. I think my longest lived fake fact was around five years. Was quite pleased with that. Eventually I got bored and moved on but it was a great game while it lasted.
>Letting anybody edit any page is a great way to get AI and human slop everywhere.
but that's not how Wikipedia works. articles about noteworthy or controversial things have activist commissars and/or paid shills watching every edit, and unless your edit aligns with the narrative they wish to present, it will be undone with essentially "because fuck you" as the reason.
its value is still considerable, of course -- there are far more neutral things than controversial ones, and most of the articles about those are fine. everything even remotely political, however, is thoroughly subverted.
Something like Wikipedia is a wonderful idea in theory, but it's tough to defend when their policies allow factually incorrect content to remain just because it appears on a third-party source that the editors consider reliable. Most media sources contain a pretty high percentage of fake news and when this slop makes its way into Wikipedia that makes the problem even worse. Can we find a better solution?
Philosiphers have been trying to work on how to define what is "True" since forever without much to show for it.
Wikipedia tries to kick the bucket by going with what "reliable" secondary sources say. Thus the goal is not to say the capital T Truth, but to essentially sumnarize academic knowledge.
To steal a saying - its the worst method except for every other known method.
If anyone has a better idea how to determine what is true and what isn't - i'd certainly be very interested.
I think the Gell Mann effect applies on Wikipedia just as much as in the papers. I’ve often come across articles in an area where I’m an expert and they have issues. I used to try and have them changed, but many years ago I just got exhausted by fanatics who seem to have just gotten really attached to their (incorrect) prose.
I understand that "truth" and "facts" are squishy philosophical concepts at the edges, but I'm talking about something much more basic. There are Wikipedia pages which are flat out wrong (by any common definition of the term) and yet the editors won't allow any corrections because the primary sources are also wrong.
One thing that might help a bit would be to tag or color code specific statements with a confidence factor. So a statement sourced only from a single trash site like the NY Times or Fox News would be red, whereas another statement sourced from multiple independent scholarly journals would be green. Or something like that.
> There are Wikipedia pages which are flat out wrong (by any common definition of the term) and yet the editors won't allow any corrections because the primary sources are also wrong.
Primary sources aren't really considered reliable on wikipedia (with some exceptions), so if such a situation is happening (and you have a source for whatever version you think is correct, since just trust me bro doesn't go far on wikipedia) and you can't resolve it on the talk page, probably time to escalate the situation.
> There are Wikipedia pages which are flat out wrong (by any common definition of the term) and yet the editors won't allow any corrections because the primary sources are also wrong.
Not saying that doesn't happen, but do you have an example of such a page?
Sorry but scientific method has been the default for "Truth" for many academic subjects. It is reliable and idempotent. If all of the human information and most of the knowledge workers are removed from the Earth tomorrow, many will arrive in the same conclusion in most of the natural sciences and maths. The language and representation will be different but prime numbers will still be prime numbers and the matter will still be made of one neutral and two "interactive" elements.
Social sciences like politics, history and economy are harder to replicate with the scientific method, however repeated observations, statiscial information as well as qualitative analysis with evidence-based research can still provide replicable analysis.
> If anyone has a better idea how to determine what is true and what isn't - i'd certainly be very interested
This kind of excuse is often cited to shut down any discussion, especially in history, politics and economics. I find it deeply anti-intellectual and dogmatic. As long as evidence and counter-evidence is followed, it should be possible to discuss alternative interpretations and solutions.
While I agree with your sentiment, I must add to this that to say that something is unequivocally true that holds in all conditions everywhere is difficult to ascertain in general. So we have to deal with that two (or more) things can be true at the same time (sometimes). This may be less the case for things such as prime numbers but more the case when the underlying principle is void of simple cause and effect (in the linear sense).
The scientific method can be used to falsify hypotheses. It cannot prove that something is true. Fields that use the scientific method typically rely on social consensus for determining the truth. Empirically speaking, the social consensus seems to converge to a reasonable approximation of the truth with a high probability, but the process can take decades.
> This kind of excuse is often cited to shut down any discussion, especially in history, politics and economics. I find it deeply anti-intellectual and dogmatic. As long as evidence and counter-evidence is followed, it should be possible to discuss alternative interpretations and solutions.
How do you propose to discuss "alternative interpretations and solutions" if you believe that doing so is "anti-intellectual and dogmatic".
I'm honestly confused how you could respond this way to my comment. It sounds like you are criticizing yourself (?)
yeah and for some reason primary literature can't be used, except when it can, but review articles are allowed. makes no sense.
Literature reviews summarise the state of knowledge in a given field, and given Wikipedia articles are about the knowledge on a subject rather than the contents of a specific paper, I don't see why it would be unusual to rely on them. Is there anything specific that you find doesn't make sense to you? It's fairly common in, e.g. medical research for individual studies to be not relied upon and meta-analyses are certainly generally considered stronger evidence on any topic.
> it's tough to defend when their policies allow factually incorrect content to remain just because it appears on a third-party source that the editors consider reliable.
I feel this is the kind of discussion that would really benefit from a number of concrete examples.
I feel like that is the kind of response that is an example that could be used for "sealioning" on Wikipedia.
> Sealioning (also sea-lioning and sea lioning) is a type of trolling or harassment that consists of pursuing people with relentless requests for evidence, often tangential or previously addressed, while maintaining a pretense of civility and sincerity
You really believe that applies here?
Its hard to say its relentless when not a single piece of evidence has been given so far.
Its hard to say its tangential when it speaks to the original poster's thesis.
I don't know it applies; but given the many times Wikipedias problems (esp politically biased gatekeepers) have been pointed out, followed by innocent-sounding requests for examples, followed by long back and forth time and energy sapping arguments with dismissals and refutations rather than thanks and acknowledgement of the problems, my instinctive reaction is suspicion towards people implying this is their first time hearing Wikipedia is extremely flawed wrt source material and editorial power.
This entire thread is filled with people saying Wikipedia is doomed due to whatever minor problem personally bothers them, in this case apparently coverage of recent events quoting news organizations.
Nobody is actually showing us why the problem they choose to highlight is significant enough to torpedo Wikipedia altogether (or even that is a more important problem than all the others mentioned).
Also, nobody here is starting a long argument to waste people’s time. They’re simply saying “your one paragraph proclamation is completely unconvincing”.
Of course it has faults, but so did the Library of Alexandria. However, can anyone argue that Wikipedia is not our best attempt at collective facts?
Of course agreeing on collective facts is extremely messy.
Show me a better attempt, please.
What are "collective facts?" How do you distinguish "collective facts" from "alternative facts?"
It is hard to imagine a worse attempt at establishing a collection of commonly accepted truths than Wikipedia. Scientific journals are overwhelmingly better at weeding out falsehoods and hoaxes, despite the history of successful efforts to dupe reviewers. University libraries are also better at collecting knowledge, and in fact the entire industry of journal publishing began at university libraries. That knowledge is not locked away from the masses, it is just less convenient to access than Wikipedia.
Even Wikipedia has had to accept that a certain amount of oversight and review is necessary. There are some topics that attract bad actors, paid shills, state-sponsored propagandists, etc., and Wikipedia had to introduce "protected pages" to deal with such things. Add a structured review process and more rigorous citation requirements and those "protected pages" would be a lightweight version of academic publishing.
Fair to assume that the library of Alexandria contained books written by the most erudite scholars and philosophers of their day, not Joe and Jane and their cohort of terminally online gatekeepers.
Why is that fair to assume? "Books" were extremely rare then. Some random librarian had to include those books, and likely leaned towards taking all written works due to scarcity.
Likely some other librarian disagreed with some inclusions and certainly many readers disagreed with the content. They must have had a system for all of this. It was far less open than Wikipedia.
IIRC, they seized all scrolls from all ships in the port, in many languages. They likely had many "errors" according to other librarians and many readers.
> can anyone argue that Wikipedia is not our best attempt at collective facts?
Maybe at free collective facts. If you want to pay: JSTOR and Google Scholar, for example; textbooks; or any of the endless scholarly encylopedias, guides, and review papers, especially on specific topics. And if you don't want to pay: shadow libraries, or the reference room of an academic library.
Well, that's a fair argument!
I am not deeply familiar with the process of peer review, but is it open? As far as I know, it is confidential on purpose, correct? Doing all of this with an open moderation trail is what makes Wikipedia truly ambitious, isn't it?
I have spoken with a few Reddit mods, and Chad, creator of automod and reddit's 12th hire about this topic. Everyone agreed that truly open moderation would be the biggest shit show ever. Yet, that is ambition of Wikipedia.
Can you imagine what Dang and Tom would have to deal with if all of their moderation was open? I might have some disagreements with HN moderation, but open moderation is not something I would wish upon them.
I guess Larry Sanger was right all along about experts vs amateurs?
Wikipedia has been the default result on Google for most things for years now.
Academics have a duty to make quality knowledge open. The difference between what is outside and inside the walled garden is astounding. They most of the truly valuable knowledge to themselves, and leave the public to feed on scraps, augmented by mountains of junk food, drowning in an ocean of sh-t (misinformation).
Wikipedia is existentially threatened by AI, which does the same thing but better.
> most users of AI products do not click the links to Wikipedia, choosing instead to accept machine-generated outputs without further verification. If this practice continues, readership on Wikipedia will continue to decline.
This is the same mechanism by which Wikipedia drained readers from news outlets and academic papers that it cites for information.
I play the tiniest violin in the world for Wikipedia editors that now suffer from the same thing they did to the news media.
[dead]
[dead]
[dead]