As one of the people who've been involved with the Gen 8 update process from the beginning, I'm going to be commenting on the entire process, from the framework we used in the PRC to Friday's livechat discussions.
Addressing the elephant in the room, Gamefreak absolutely fucked us this gen in regards to CAP updates. The original PRC was started far before Dexit and its effects on Pokemon were known, meaning that the original framework snake provided needed to be updated to accommodate the massive changes Gen 8 brought to move distribution. The problem here was that there was an extreme lack of information about a lot of process critical components leading up to and during Gen 8. The dates for DLC release had to be and are still speculated. Information for Pokemon Home, specifically the interaction between it and transfer moves, were entirely unaddressed until the weeks coming up to Home's release. What moves were unusable and what moves were simply unobtainable had to be datamined directly from the games themselves because an official list wasn't available. In short, Gamefreak broke all trends we relied upon when making the original framework and communicated next to none of these changes clearly, which meant that the CAP mods had to try and figure out an appropriate response both in the thread and in private channels from the incomplete information that was available.
Dexit and its associated move removals and the introduction of a TR system were beyond anything we could have anticipated. When I was helping snake draft the first update framework, we relied heavily on the trends seen in previous gens to cut down on the workload. We expected to be discussing maybe ~120 moves between new Gen 8 moves and the established 100 TMs, and this was reflected in the streamlined timeline we presented. This was already a too large a number to discuss, which is why the framework made a heavy emphasis on pure flavor additions in order to avoid lengthy discussions over balancing. Before Dexit, I honestly expect Gen 8 updates to have us looking at maybe 20 relevant moves in total for discussion per mon. In reality, we had 200 moves split between TRs and TMs to apply to 27 Pokemon, on top of the new gen 8 moves, on top of all the moves that were removed from the game, on top of all the moves that were now gated behind past gen TMs and tutors. To further complicate matters, details on Home were nonexistent, so we had to assume the worse case scenario that transfer moves would not be usable. What started as a speculated ~120 moves for discussion blossomed into at least triple that amount between the newly distributed moves and the currently unavailable moves we had to discuss.
Obviously, more moves to discuss meant that there was a higher workload than expect for the updates. Unfortunately, this was not accounted for in the final PRC decision. We went with the timeline snake provided for the original framework, which was seven ULs hosting seven threads at one week intervals. To put this in perspective, if we divided thread time evenly between discussing move removals and move additions, that was 3.5 day to discuss either topic for 7 different Pokemon. In hindsight, its really no surprise that our update project dragged on for far longer than expected, as the initial expectations far underestimated the amount of material we needed to shift through. This resulted in an under staffed, poorly motivated, and overworked UL team that suffered from some pretty intense burnout by the second half of the project.
Going back to Gamefreak and updates, Home was a big kick in the teeth for us. The workload before home was roughly double than the workload we had after home, as the process went from discussing removals and additions to solely discussing additions. This was honestly a motivation killer, as Home invalidated a ton of work that went into previous update threads. If Home didn't allow transferable moves to be used in Gen 8, I doubt that Friday's livechat would have been as productive as it had been if we still had to discuss removals.
Now that I've addressed how we poorly the planning stages went, lets jump into the actual project.
The first point of failure for the project was the UL nomination thread. Going into updates, CAP was at a lowpoint for user activity and motivation following the depressing last several months of Gen 7 followed by a deep dive into the cluster fuck that was Dynamax at the start of Gen 8. We needed 7 people to lead updates and we got exactly 7 volunteers capable of doing so, which was the bare minimum we needed to skate by. This also meant that we had zero substitutes set up to help cover for the ULs.
This ended up crippling the project as reachzero fell off the face of the earth before finishing their first Pokemon, meaning the UL team was understaffed almost from the start. UL participation ended up being one of the major stalling points for updates as the process progressed, as half of the remaining team were swamped with IRL responsibilities that severely cut back on the amount of time they could dedicated to the process. As there were no substitutes for ULs, not even runner ups as we had an exact seven volunteers, work piled up until updates were considerably behind.
Continuing with the theme of an overworked UL team, UL workload was larger than anything the PRC alluded to. Not only did the UL team have to complete four threads individually to discuss the massive amount of movepool changes this generation brought, they were also responsible for managing update policy, helping other ULs with various issues, and coordinating with each other to ensure the responses to each thread was consistent. The UL team ended up having to juggle responsibilities between their own Pokemon and eachothers.
Moving onto the thread execution, the past livechat discussion made it clear that it was possible for the ULs to outsource much of the busy work to the greater CAP community. Highlighting movepools, rebuilding movepools, checking egg moves legalities, etc. could all have been outsourced to the regular contributor in order to lessen the workload for the UL team. At certain points, the UL team did delegate work for the IRL busy members, but it was mostly to other UL members.
Another problem that was prevalent throughout the update process was that the community itself was hesitant to engage in some threads. Some Pokemon saw heavy discussion, other times a UL had to try and create an ICC for a move based off four posts, half of which provided bare bones reasoning. This process needed an active community to take a lot of the debating and fact checking duties away from the ULs; the goal was to let the community sort out the various issues with the ULs as supervisors. In reality, the ULs were forced to make decision after decision for sometimes arbitrary reasons because there wasn't enough feedback for us to effectively do our jobs. This put a lot of stress on the ULs and lowered the quality of updates overall, while also forcing ULs to extend their deadlines to accomodate for the low posting activity.
So, to sum up the issues I had with this update process: Gen 8 and Pokemon Home made it difficult to properly prepare for movepool issues. We failed to correctly evaluate the workload each UL member would be responsible for. We didn't take into account the low motivation our posters had or the amount of burn out the UL team would have. The community was shy to fully engage in thread discussion, forcing ULs to go over their prescribed deadlines in the hope of seeing issues resolved. The UL team was understaffed from the beginning and had very little support aside from other UL members. Work was not effectively outsourced even when it was apparent the UL team was overworked. Substitutes for inactive ULs were never found.
Now, commenting on the discord discussions:
1. I absolutely loved the discord discussions. I loved how organized discussion was with multiple channels. I loved how the UL team was out in force to ensure everything went smoothly. I fucking adored how people stepped up and created a support team to deal with movepool construction while the ULs lead their respective discussions. The livechat discussions were the definite high point of the process and I felt like they were immensely successful. We had a goal and we achieved it. I enjoyed discussing and debating with everyone which is all that I could ask for.
2. I think livechat discussions are the way to go with updates. For the sheer volume of Pokemon and moves we had to discuss, discord was the most efficient system for rapidly discussing topic to topic. Compared to forums, the pace of discussion is faster and its easier to get a consensus when you can literally ask the chat if they support a decisions. Forum threads were tedious to make and maintain and often times the quality of posts made discussion poor; discord side stepped that entirely. Moreover, the workload for each generation is only going to increase as we release more CAPs, so we need to use the most efficient form of communication to ensure updates don't drag on.
3. Was my voice heard? Absolutely, but that's to be expected when I'm a UL. For the mons that I wasn't the UL for, I still felt like my opinion was being heard and I can't think of a single instance where I was outright ignored. I ran my mons on a move to move basis, where I brought up a move for discussion and made a decision based off chat feedback before moving onto the next. I believe that my chats were structured and benefited from it. By doing so, we were able to go through a dozen moves in less than two hours, which I viewed as productive.
5. I think the biggest downside to the discord chat was that it gated participation behind availibility. I default to forum posting as my go to place for CAP discussion because it allows posters to reply to eachother across differing timezones. Discord does not allow that due to the rapid pace of discussion. Some potential ways to circumvent this in the future is to create a schedule ahead of time detailing when each Pokemon would be discussed so people can plan accordingly. Hosting livechat discussions alongside an open forum thread would allow users to give feedback on the mon without being gated solely by availability.