Forum:SH:QuickAnswers

This page is an archive of a community-wide discussion. This page is no longer live. Further comments or questions on this topic should be made in a new Senate Hall page rather than here so that this page is preserved as a historic record. —spookywillowwtalk 19:09, 21 April 2024 (UTC)
Forums > Senate Hall archive > SH:QuickAnswers

Hello friends. It's time for us to have a Discussion about QuickAnswers—a new Fandom feature being rolled out. This (above) portion of the SH will solely be the presentation of some stuff about it and some discussion points, and I'll leave my own personal thoughts below. Our Community Manager, Spongebob456, will also likely pop by into this to help with answering questions.

I'll state this also relatively up top for those who would like a tldr; essentially, this feature has, technically, been live on Boba Fett and one other Wookieepedia page for some time. Those answers were revised by admins as part of an early test of the feature. However, the feature is now rolling out to more wikis, likely including ours. We do have some questions on the Dashboard already, which're currently going to have a publish date of June 4th, ultimately, so we have a bit of time to figure out what to do about that. But it does essentially have to be discussed in some form, as it's currently half-live.

So, what is QuickAnswers? If you scroll to Boba Fett just above the Biography header while logged out, it shows up as a set of five questions (screenshot). These answers and questions can be edited by admins, rollbacks, and discussion mods, and then published as such.

Pros:

  • Increased pageviews as shown by trial wikis.
  • Helps Wookieepedia go higher on search results (SEO).
  • Studies on it thus far show some readers like to see questions answered in an efficient way without digging through the page.
  • Can customize questions and answers fully.
  • Questions that are reported enough times do get auto-pulled.

Cons:

  • At the moment, the interface only allows administrators, content moderators (which we don't have), rollbacks, and discussions moderators to edit them. There's been chatter that hopefully someday they'd make it a unique usergroup that can be given out to more users, but it's not implemented.
  • A photo of the dashboard. The questions do forcibly auto-publish at a certain date if not manually published by then.
  • WP:NPOV. This, of course, can be corrected, but serves as an example of an unpublished one. All of the original inputs there are generated by AI.
  • Questions publish at a set rate released at intervals of the community's choosing (if that's weekly, monthly, or every fornight etc.) It could be a low rate, but could become backlogged over time if not many admins/rollbacks help moderate them. The rate at the moment is 10 pages per week, 5 questions per (50 questions/week total).

Discussion points:

  • Interface management is part of WP:A already, but if adopted, would need to be part of Wookieepedia:Rollback as well.
  • If adopted, we'd need to discuss whether to default to canon or Legends answers, whether to discuss answers if they differ from both continuities, and whether to allow mixing of OOU and IU information if the auto-generated answer has it.
  • If adopted, would need to discuss a game plan for having these be moderated properly so something WP:NPOV doesn't publish.
  • The feature is intended to be helpful, but has some cons. If we choose as a community to petition to not have it, then we should start a CT either way.

—spookywillowwtalk 10:58, 11 April 2024 (UTC)

Discussion

  • The above was the presentation of thoughts as orderly as I could get it. For my own personal thoughts—conflicted. It has potential, since we can change both the questions and the answers, it means we could highlight useful questions with thoughtfully written, high-quality questions from trusted users. It really does have a lot of potential, especially because, though I dislike AI personally, we could 100% rewrite everything to be human-done. However…due to the forced publishing timeline and the current rate of questions, we would definitely struggle severely to be able to keep up. Even with the growth of the rollback team recently, these do take a significant amount of time to rewrite. If it had no timeframe at all, then perhaps we could write them at our leisure and make them as high-quality as Wook is known to be. I do however, firmly not want unmoderated AI content being pushed to our readers; it's quite biased and flawed in its current state and we have a lot of people relying on us. It also unfortunately is currently set up for only some Wookieepedians to be able to edit at all; thus burdening admins and rollbacks, whom already do a lot of paperwork (or board paperwork on top of admin paperwork). But nevertheless, it's a feature come our way that's being implemented, so should it end up sticking, we'll need to make guidelines for it per above. And it's definitely worth an open discussion too, at the least. Perhaps if Wook did have a larger 'rollback'/QA moderator team, it'd be manageable to actually moderate these, and then in that case, it might be really lovely to have on some status articles.—spookywillowwtalk 10:58, 11 April 2024 (UTC)
    • Already discussed some of Spooky's concerns with her on Discord but just want to acknowledge here that they've been seen. --Spongebob456 talk 14:38, 11 April 2024 (UTC)
  • I've spent quite a bit of time thinking about this and agree with a lot of spooky's points. Ultimately, I'm confident that this isn't something that we should use as a feature in its current form. Upfront, I want to say that no matter whether you agree or disagree with the feature that we should all keep discussion respectful regarding the people at Fandom, especially keeping in mind that people like Chris didn't come up with this and are simply passing along information to keep us informed. As the feature currently stands though, it poses risks to our reputation as a trusted source of information as well as having potential to cause burn out among some of the most active members of our community.
The AI questions I have looked over so far have real problems in tone, factual accuracy, and are unable to differentiate canon and Legends, a big issue specific to us. Should they be auto published in this form, they would not only misinform the reader but also make it obvious AI has been used based on tone. This would result in backlash from readers on social media against our editors, as it would likely be assumed we were responsible, and would also potentially cause people to question the reliability of the articles themselves or if we had used AI there as well.
Since publishing them in this form is clearly something we would avoid, there is then a burden of work placed upon those who can edit the questions with a deadline if we want to avoid the risks mentioned above. Those with access, whether we choose to expand it beyond admins/rollbacks or not, will certainly be among the most active editors on the wiki and this will not only divert their attention away from working on other parts of the site but also likely cause burn out. Everyone here is an unpaid volunteer and being forced to work to a deadline on topics we haven't chosen with negative repercussions for the whole wiki if we fail will be draining for all involved. The more people drop off from editing questions, or editing altogether, the more burden there is placed on whoever remains.
I understand the benefits of SEO and how these will help with that, but Wookieepedia already has extremely strong SEO from my understanding and so with that being the major benefit to these questions I think the cons heavily outweigh the pros here. If this were reintroduced as a feature without any deadlines and with questions only ever being published after users have chosen to review them then I'd be far more open to it, but in its current form I would strongly encourage everyone to vote to reject the feature in any upcoming CT unless changes are made. Ayrehead02 (talk) 11:06, 11 April 2024 (UTC)
    • Already discussed some of Ayrehead's concerns with him on Discord but just want to acknowledge here that they've been seen. --Spongebob456 talk 14:38, 11 April 2024 (UTC)
  • I echo spooky's comments. The purpose of QA is to have answers to common questions that can be easily picked up by Google's search results, but the AI-generated writing is Not Good. It would be a nice feature if more users are allowed to write up the QA boxes and build them up organically instead, at our own pace. OOM 224 (he/him) 11:10, 11 April 2024 (UTC)
    • Passed this on! I know we can certainly be very flexible on the review period and number to review if it helps. Making the review period indefinite may be trickier as this may create a bit of a backlog of questions as they come in. Re more users, watch this space, seeing how we can go about that. --Spongebob456 talk 14:28, 11 April 2024 (UTC)
  • I think in concept, QuickAnswers could be a useful feature for readers. However as discussed above, I am very much against having any AI-generated content on mainspace articles. The ones so far are poorly written, violate NPOV, and often just make stuff up. Rsand 30 (talk) 11:38, 11 April 2024 (UTC)
    • Hey, this has been passed on. I know we're looking for ways to keep improving the language the AI uses. This is why the Dashboard is useful if some answers need some rewording (but I get this is still manual moderation). --Spongebob456 talk 14:28, 11 April 2024 (UTC)
  • A question Chris can hopefully answer. Since we have concerns about Legends/Canon splitting, can the QA differentiate between the two? Right now Darth Plagueis has some QA pending. As I understand, those will appear on the canon version of his article (Based on how I'm seeing Boba Fett's article act) but given his lack of Canon appearances and major Legends appearances, I feel people will want QA about his Legends history more. Is there a way to adjust that? NBDani TeamFireballLogo-Collider(they/them)Yeager's Repairs 12:44, 11 April 2024 (UTC)
    • Hey - I've passed this on to the relevant people to look at. They are looking for a solution to differentiate between the two but I don't feel comfortable right now to say definitely that we can or cannot ensure that. We can say for sure that we can differentiate with Fanon content, but this is a different setup, so we want to investigate properly. Will keep you posted! --Spongebob456 talk 14:28, 11 April 2024 (UTC)
  • After carefully reading the Fandom blog on the subject and this SH, I am fully aligned with the position of spooky and Ayre. Also, I would like to ask... what is the deal with "Fandom Trivia"? I'm only noticing it now because I am always signed-on, and it seems to be no more than three years old, and I can't seems to remember any discussion on the subject nor can I find any topic on the SHs archive on the subject, which is kind of worrying to me. NanoLuukeCloning Facility 13:30, 11 April 2024 (UTC)
    • Are you referring to the module on the Obi-Wan Kenobi page that's viewable while logged out? This was part of another engagement initiative we ran in 2021/2022 focusing on quizzes. Admins have the ability to edit these and indeed worked on them after the initial round of testing. Admins can also target/untarget these to pages as they wish. Quizzes are still a thing and new ones can be made, and they have shown to be effective in terms of keeping readers on the site. In terms of why SH's weren't made, I'll leave that to the admins to answer as that's more a local concern. Hope this helps. --Spongebob456 talk 14:28, 11 April 2024 (UTC)
  • As a Rollback, I was briefed on this ahead of time, and my impressions weren't good, especially because of the AI element, but I intended to accept it at least until it began being rolled out. Having seen some of the preliminary Quick Answers...yeah no these are not good. I am frankly very worried at the idea of low-quality AI-generated material that doesn't distinguish between Canon and Legends or present a fully professional, neutral tone being pushed to our readers under our site's 'brand'. Our readers are perfectly capable of looking for information like "Did Commander Cody survive the Clone Wars?" and "What was Baylan Skoll's starship?" themselves-the point of a wiki is that fans are able to dig around looking for what it is they're trying to learn. As Ayre said, each user here is an unpaid volunteer, and those of us voted to have rollback or admin permissions are still volunteers who are now being essentially asked to choose between doing the work of the site, or moderating the low quality output of a feature pushed on us by Fandom. My final point is that I am inherently concerned about, and opposed to, any usage of AI to replace or supplement the human editors who actually care about the content of this site and are, again, unpaid volunteers whose work provides Fandom with the content their business runs on. TLDR: I have many ideological reasons to oppose this in concept, but both having seen the early versions and as someone who would be expected to moderate this, I can not say I support having it on this website I am so proud to be a part of. Fan26 (Talk) 13:44, 11 April 2024 (UTC)
    • Hey - just a note I answered above re Canon/Legends distinction but upshot is we're looking into that, will keep you posted. Re "...pushed on us by Fandom" - just want to say nothing is being forced here. The idea is we have been working with wikis on this, but if there are serious issues, let's talk about it. Staff are actively investigating issues being raised - if you're able to wait it out and give it a shot, great, but if not, we're not going to force it. We want this feature to be beneficial to wikis and numbers are showing it is, but clearly we want it to be workable for editors. --Spongebob456 talk 14:28, 11 April 2024 (UTC)
      • I'm glad staff is actively listening to feedback and intends to work to improve this feature, but as I said I do have issues with this concept as a whole for a multitude of reasons. —Unsigned comment by Fan26 (talk • contribs)
  • I am not a fan of this. I'd like to avoid potentially-inaccurate AI output disrupting our reputability, especially the NPOV and Canon/Legends problems that others have touched on. And speaking more generally, I would not want AI to be at all associated with any of the content on Wookieepedia. AI content on the site would, in my view, even as a supplement, begin to tarnish the efforts of editors past and present. In terms of editors being the ones to write the answer the questions, I don't imagine that many of the active editors would think that is a good use of their time. I don't want to be too harsh because I think this feature has merit as an idea, but I am personally not in favor of it. Wok142 (talk) 19:15, 11 April 2024 (UTC)
  • Conceptually, I'd be potentially fine with this as long as it's tweaked. In practice as it is, I am against it—ESPECIALLY while it utilizes AI generated content. Absolutely not. - Thannus (DFaceG) (he/him) (talk) 19:41, 11 April 2024 (UTC)
  • Just echoing the above thoughts, this would not be a good move for the site unless serious changes were made to the whole concept in general Lewisr (talk) 04:47, 16 April 2024 (UTC)

Staff update on QAs

Hey all! Thanks for the detailed feedback above. I've passed this SH thread to engineers and they can see early on where QAs are falling short for Wook. In light of this, we've pulled Wook from this QA test while we work on improvements to the feature. Feel free to reach out to me if you have any questions. Thanks again! --Spongebob456 talk 13:37, 16 April 2024 (UTC)