eylul: (Default)

[personal profile] rodo's post few weeks ago prompted me to finally begin typing out some of the issues around OTW and AO3 I’ve been thinking on. The topic of how to fix OTW in terms of maintaining the volunteers it has and attracting more has been brought to forefront during last election, and a lot of thoughtful ideas have been going around focusing on different aspects of the org before and since. Apparently, even the board has finally begun their own conversation, even if it is behind closed doors for now.

Out of the discussions that are visible at least, some I agree with, others I don’t. One thing that is constant throughout all of these however, is AD&T (Accessibility, Design and Technology) committee which is currently the main decision point for almost everything related to AO3. Solutions are formed around its existing makeup, maintaining its fixed presence even though it sits at the heart of a cluster of structural issues. I think it is time to voice it however, that AD&T itself needs to be reformed and reorganized.

So rewind, who am I and what are my qualifications? My name is Eylul and I am a staffer at the Organization for Transformative Works. I joined OTW toward the end of 2009, shortly after the open beta, as a design and testing volunteer. I was part of the support subcommittee (before it became its own committee) and was involved in writing of its original training manual, I was the lead of the QA (Quality Assurance) subcommittee (which is also informally known as testing subcommittee) for 2010, and then I&O (Internationalization and Outreach) liaison of AD&T in 2011 before taking a break from the committee and nearly decided to quit the OTW altogether. I am currently an I&O staffer, and involved in the Survey and Category Change workgroups. Outside the organization I am a Fine Arts masters student starting her thesis. However I actually completed my undergraduate degree with double majors in Computer Science and Media Arts and Sciences, and done grad course work both in Computer Science and Arts. While I have made a career change away from tech field in general, I have work experience in customer support, and research experience in HCI (Human Computer Interaction).

What is the context? AD&T is short for Accessibility Design and Technology. From the name, it can be inferred that at the inception of the organization, it was meant to support many projects of the org that might have various technical needs. Except somewhere along the way AD&T became the AO3 committee in practice, solely limited to development of one project and none other. AD&T is also currently responsible for non technical aspects and issues of AO3. Decisions on strategic questions such as which features should be prioritized takes place within AD&T, even though such decisions would ideally take in more points of view. AD&T also coordinates decision-making on AO3 matters when other committees are involved. Not only all this is a huge burden on a single committee but it causes considerable conflict of interest between supporting volunteers of different roles, supporting users of the project and supporting OTW’s goals.

As it stands AD&T committee, which is almost exclusively formed of coders, wears several hats ranging from being the design process itself, to overseeing the AO3 project management, to supporting coders and testers, and to coordinating inter-committee issues related to the archive.

What do I propose? I think that the management of the AO3 as a software project, and management of technical resources of OTW needs to be completely overhauled, and propose following two changes:

  1. Dividing the current AD&T into 4 committees:
    • Design
    • Front-End Coding
    • Back-End Coding
    • QA
    All of these will serve the whole OTW as needed instead of just AO3, similar to the way Systems and Translation committees currently work, although it is likely that some volunteers at least will have specific projects they prefer to focus on.
  2. To form an AO3 committee that will make the decisions on archive priorities and production that is based on advice and input from all parties, and coordinating the work from various committees.

Below, under the cut I will explain my reasoning for this proposal under several sections: On first two sections, I will explain how separating committees based on skill resources from those focused on projects can benefit OTW and AO3. Then on the following three sections I will focus on four skill based committees, and how volunteers,the AO3 and the OTW will benefit from this new structure. Then I conclude with why this restructuring becomes more and more necessary as the AO3 matures.

Accessibility, Design and Technology to All (Projects of OTW)

Currently we have several projects within the org that are in need of technical assistance. The website has Webmasters, but for example the wiki forums were, until recently, actively seeking a coder. This is a crucial project since it is in part a test case for OTW wide forums, something the board member [personal profile] julia_beck has been pushing for a long time now. In another case, Vidding’s Dark Archive and Torrent of Our Own cannot get off the ground in part because it is not only coding that needs to be done, but also the support structure around the project for the volunteers.

In short, we are reinventing the wheel of training, organization, and management for each starting project. Not only that but each project is intended to have a dedicated QA team, Design team, Support, etc. This is a huge overhead for an organization of OTW’s scale. Considering how long it took AO3 to establish some of these procedures, it will be a huge loss of time spent in inefficient procedures until they can be polished. Another drawback of this issue is that often these start-up, or short term projects have few technical volunteers in them. Keeping them to their own cluster takes away existing resources as training materials, mentoring, and brainstorming within a team.

I am not saying technical volunteers won’t specialize in projects. Some people will want to specialize. However if they do not, they will have the same structure backing them regardless of which project they decide to put their energy to. For the organization this will bring flexibility because it will be easier to start up new projects and training new people for them, and volunteers will still be able ask advice to the rest of the pool, and request temporary help or mentorship.

A note here. I know there are people who feel AO3 should be the only focus of OTW and that no resources should be wasted elsewhere. If you feel this way, you might feel this would just mean more resources taken away from the production of the archive. I would expect a good portion of the volunteers will still be working on AO3 for now. What is offered here is to divide the management work of the current AD&T into specializations, then allow some of these specializations to be also available to rest of the OTW projects. This division of work will actually also benefit AO3: the rest of this long post focuses on how having 5 committees instead of one will benefit the project.

It is also important to note that the OTW projects are an ecosystem. OTW Forums will allow the AO3 related feedback, like all other discussion, to happen in a space where it can more directly affect the outcome. The Dark Archive and related projects are directly related to vidders using the AO3 to link and showcase their vids. The examples can go on.


Archive Committee Rather Than AD&T: Dedicated Project Management

As I briefly mentioned above, most of the decision making on what to prioritize to code are made by coders (and testers to a lot lesser extent), since they are the ones who form the AD&T committee. It is true that requests for advice and delegations to other committees occur thanks in part to AD&T members taking the initiative. It is usually limited however to content or to design of features that will be directly used by another committee, such as tag wrangling. It is also true that there is a roadmap in place. However, how much the roadmap is put into consideration if at all, and what considerations were made when it was decided, are questions to ask.

Thus, while the status quo sort of worked enough to not burn AO3 down into cinders, it resulted in some pretty bad prioritization errors. Today, AO3 still doesn’t have a translation interface which loses us non-english speaking fans. The support board is an interface that will replace the current form with an interface that will allow users to help each other, and allow on better dialogue on requests of bug fixes and features. This feature which was proposed over a year ago, which will benefit not just the support team but the whole archive development, is also still absent. Both are pretty critical projects. Similar prioritization errors are not limited to major new features. Fixes and improvements ends up being prioritized based on if AD&T members notice its effects directly in their circle or are themselves affected by it. The problem is not the faulty judgement of individuals here, it is that it is not, and should not be, coders’ job to have the final say in what needs to be worked on most urgently. It results in a conflict of interest.

There are 3 main issues here

  1. As much as coders and testing lead (AD&T recruits staff most heavily from coders, followed by testers; it's relatively rare for someone to be in the committee unless they're a coder or were staff elsewhere in the org.) will try to take initiative to seek output, they will hit bias. Currently, when it is not 502 errors or a similar urgent issue on the site, coder preferences influence strongly what new features come to the archive, rather than roadmap or goals of outreach or any other reason.
  2. Coders and testers being actively used at managing work is a huge waste of the scarce resource that is our technical volunteer pool. It is also a bad waste of our existing volunteers who are talented in coordinating and managing.
  3. That AD&T becomes the key point of decision causes a lot of interest in being in the committee. This can result in a lot of people at a time who would rather do what they are in the org to do, rather than talk about topics that are not their interest OR expertise area but sticking with it because they want to have a say. It also results in people on other committees who should probably pitch in to not have a say in the final decision.
The solution to this is a smaller archive committee setup that takes advice and reports, similar to the way the strategic planning workgroup currently functions, in order to decide what to prioritize, to coordinate, to set goals and follow up on them.


Design as its Own Committee

The current design process of AO3 is a combination of two approaches. In some cases, a coder just produces a feature, with little to no planning or documentation and showing it around in next AD&T where there is time. Other times, a design is discussed in the AD&T committee itself.

The problems with this process are several

  1. Since there is no design process,the decisions on new features are often made by discussing them in the committee. AD&T tries to alleviate this through their own initiative, looking to see if there are support requests for features. However, often if a senior coder wants something in, and are willing to code it themselves, unless a drawback is obvious, it will happen. There are no user models, no target audience documented. Nobody really thinks what the workflow of a user will be until the work is nearly complete. This contributes to the prioritization issues mentioned above and also results in UIs such as the Post new form, or collections form that require videos to explain.
  2. If you want to design, you need to be part of the AD&T committee, which implies unless you have been designated as a QA lead or Design lead, or another position by the committee, you need to be a coder. This closes the doors on people who wish to focus on usability or aesthetics and especially those who specializes in them on expert level, who might not have the time to commit to be a coder in expected time commitment AND design.
  3. AD&T assumes the role of design committee itself. As mentioned above, the committee does tons of other work, varying from discussing code optimization, to coordinating content changes that comes from the workgroups, to release management, and to project management. Putting everything aside to discuss a set of icons half an hour in a meeting, or spending hours of time on it over emails is a very inefficient way to do work. Brainstorming is necessary, but it is the job of design volunteers. Not of coders, not of testers, not of higher level AO3 management.
  4. Design by coding is a waste of scarce resources of coders. Essentially, design is a process where you want to go in several iterations going from quick to slow, adding more details and fleshing out at every step while making sure you are on right track. The designers start with a workflow, then UI, the testing beginning as against a user model, then gradually involving active test users. In situations where the design team can interact in physical space these first iterations are actually done by sketching on paper. While we have to adapt some of these early stages to our setup (the organization doesn’t meet in person), we can have the iterations, brainstorming, acquiring feedback at several stages during design, and coding from the user base. Thus by the time it gets to the backend coders they know what they are coding, and half of the code doesn’t suddenly get obsolete midway through because of settings not reflecting the common uses of the target audience, or a problem of a similar nature.

    Note that, it can be/will be desirable at some point for the designer to prototype, but the designer shouldn’t have to understand the intricacies of the archive code to do it. Just some HTML and javascript to mock up independently should be enough to decide, document and pass on to the front-end and back-end coders.

An independent design committee combined with an AO3 overseeing committee will allow solidifying the use of design process within the archive. The description I gave above mostly exemplified usability design, but visual design, illustration, data visualization are also areas we have/can use people in. Making sure we have work to do for these people, and provide them support so that they can do what they do will benefit the archive.

Generalizing the design team will benefit volunteers doing such work in all projects. Especially those working on OTW Website or Fanlore. Also some aspects of design are required on less apparent places, like graphic design in case of production of print media and items (print media, donation gifts, etc) or data visualization in case of Survey workgroup, or Communications when they need to put out statistics of any kind up. Volunteers working in these spots can also benefit from the existence of a design team with its mentoring, training and support resources.


QA (Quality Assurance) as its Own Committee

The QA’s role is to make sure that the outgoing code is truly ready for the users. It is not only sufficient that it is bug free but is also that it is of sufficient quality, in terms of functionality and design. AO3’s current QA process is realized by a subcommittee of AD&T. This subcommittee is generally known, both within and outside the org, as the testing subcommittee, despite it being clear from older documents that they were intended work as a QA team. A subcommittee within the org, as well as outside is generally known as testing committee, despite it being clear from original documents that they were intended for former. (The difference between QA and testing is out of the scope of this post, and it is sufficient to say, at the current scale of the AO3 and the org, it is perfectly acceptable for one committee and set of volunteers to have a workflow that covers both roles)

Again several problem points.

  1. Testers being on pressure to test code that is delayed already on a strict deadline. Therein lies one of the biggest reasons for testers attrition. The monthly release schedule and automatic test suites helped somewhat but not enough. The workload can be described as long breaks where nothing happens punctuated by days of allnighters trying to finish everything to a strict deadline.

    This can be in part alleviated by TDD (Test Driven Development). This essentially means coders writing a series of automated tests before beginning to code, based on a design and documentation. [personal profile] jennyst who is a board member and a former AD&T staffer, has a post about this process. It doesn’t stop the need for manual QA, but it does lessen the risks of all sorts of small bugs and mistakes making to the test release. It also has the added benefit of having documentation in form of test spec which is crucial for manual testing process to be healthy. (more on this in the next item)
  2. Lack of documentation, which ties into the design issue. You can’t just code something large on the fly. The consequences of this for one is that often it is not clear what the QA team needs to test, for the coders to have no clear idea what they are coding in first place. In AD&T process, testing volunteers need to often ask coders what they meant to do which is both problematic in terms of time requirement to have the coder present answering questions, and it brings the bias of the coder’s point of view of the function or feature to the QA volunteer, making it harder for the QA volunteer to catch buggy cases, inconsistencies or other issues from an independent point of view.
  3. Lack of QA’s input in design and usability bugs. Just because something is working doesn’t mean it isn’t convoluted. Since there is no process to work on this, usually QA volunteers stumble into these. However there is no structure to bring this feedback back into the system in a way that will result in a fix other than the QA volunteer or QA lead personally convincing the coder that there is an issue.


One of the biggest problems in AD&T is that like design, QA remains an afterthought, rather than being truly part the process. It is not very surprising knowing programming culture in general and the make-up of the committee. Splitting the QA into a separate committee and having them answer a separate managing committee rather than coders, whose priority is naturally seeing their code released as quickly as possible, will help to solve a conflict of interest that arises not only between QA and coding teams, but also between coding teams and users, who might be affected by a bug or low quality update.

Separating QA also allows us to build a structure within the OTW where the team can serve other projects. Website, wiki, new projects that the organization is planning are all cases where QA is needed. For example during a recent upgrade by Systems on several databases, volunteers doing the QA for other projects to ensure nothing was broken had much less resources (docs, training, etc) to work with. As I mentioned above, the OTW projects are an ecosystem. Making sure we have a set of projects benefiting each other, and adhering to a standard of quality will return as a benefit to the AO3.


Back-End Coding and Front-End Coding: Reforming Our Coding Process

In the current structure of AD&T, coders all answer directly to the AD&T committee and are organized by them. There is no coding subcomittee. Coding volunteers are divided into two groups based on skillset.

  1. Back-end coders, who mainly work with ruby, functionality of features, database access, performance etc.
  2. Front-end coders, working with HTML, CSS and javascript to reflect the existing functionality of the site to the users, in a browser compliant way.
The current coding process for both groups is as follows
  1. beginner coders takes smaller, easy to work on bugs
  2. intermediate coders takes over small features (with a mentor if possible) or bigger trickier bugs.
  3. senior coders takes over whole features by themselves, and pick up everything else anyone in above 2 categories that cannot do on their own.
1 and 2 are okay. 3rd is recipe for disaster. A lot of these features such as collections, tags, UI redo, are extreme amounts of work. During my 2 years in AD&T I wondered when this workflow would explode on our face. It finally did with the new UI last fall, resulting in criticism of unfinished layout that was not accessible to some, and offensive skins. The problem wasn’t that the front-end coding there didn’t need fixing. It did, but 3 major features (yuletide upgrades, front-end code streamlining and new skins) were piled up on a crammed deadline, and 2 of those features were done by one person. This structure results in both overworked (and in consequence defensive to criticism) coders, and volunteers lost because of having to choose between the options of spending more hours than any non-profit can realistically expect from a volunteer or being stuck with doing beginner’s work regardless of their skill level.

An obvious question is why is the work not shared. This is not possible when there is no set design and essentially one feels they need to take over the whole the project and go with it because of the misconception that anything else will take too long and that it is the only way their vision on how the feature should be will be realized. Furthermore at least some of our coders, having no experience in working as a team on same code, are probably not aware of how to use and write documentation to collaboratively code, and thus in general feel that it is impossible to work together without the work taking a lot longer. There is the additional issue that, without a proper design process and documentation it actually MIGHT take longer.

One of the major advantages of coding being split from archive management, and all of the other overheads such as design and QA issues, is that there will be a managerial position who will have time to reform our coding process and maintain it. There will be a dedicated structure to set up training, and to support coders through various difficulties they might be having that is causing a delay or stress to the volunteer. There will be also a dedicated setup to follow what happened with individual tasks, to assign more people or mentors, and to ensure that coder related documentation (specs and APIs) are done and are up to date.

Having the coding committees split will also allow it to serve multiple project that will allow support of volunteers doing technical work on wiki, website and future Dark Archive. One question that might be asked is why to split front-end coding and back-end coding into two committees (Front-End and Back-End Coders). The way I see it the new Front-end coders would be a merge of technical volunteers of the current Webmasters committee which handles the OTW Website (and is a committee overburdened with double roles of project management and development as well) that does a very similar job to front-end coders of AO3. Essentially they both work with html, css and javascript. They also both deal with a lot of challenges unique to browser-side coding such as having to follow quickly changing technologies (browser side technology has a tendency to be a lot more volatile than server side), writing code compatible with various competing browsers at once, as well as be up to date on SEO (search engine optimization) techniques and accessibility standards. On the other hand back-end coders need have their own challenges such as handling server loads (think of the 502 errors due to too much traffic), that front-end coders don’t need to deal with. Briefly put these skill specializations are different enough that allowing different committees to focus on their priorities and issues separately, will improve their efficiency and support of the volunteers.


AO3 as a Product versus Experiment

If you have read through this whole thing so far you might at this point have either one of these questions.

  1. why do we need this extra setup when it has been working more or less
  2. why wasn’t this done the way it is from start.
When starting a new website or software it is acceptable to start with less organization. Especially if the company, organization or the team behind the said website or software is not necessarily established. The team is often small, there is a strong need to just get something up to see if the idea will work, or if it will hold, or to keep the momentum going. I can see in a beginning environment that AD&T as one committee was enough and that the focus was on getting the code out as fast as possible. It was an experiment. There was no guarantee that it would work.

Even when the open beta happened (which is right around when I joined the OTW) the archive was still an experiment. People tried it yes, and received accounts, but it was more of an indie project people fiddled around than used daily.

At some point through this changed. Today AO3 has over 60.000 registered users (and a lot more unregistered ones) some of which use this site as their primary fannish venue for activities such as posting work, running challenges, reccing, reading or commenting. At this point where people rely on uptime and continued quality of the website, the AO3 stops being an experiment and begins to be a product and needs to be treated as such.

What is the difference between a product an experiment in these definitions? Basically if you want to make a product you need to think about who you are marketing it to. There is a lot less flexibility for risky ideas, a lot less leeway to break things (no matter if you call it version 0.n or 1.n). There is the expectation to fix problems in reasonable speed, and to "not fail".

Being a product is something a lot of open source software fail at. Actually it is something even smaller companies fail at. It is the difference between being various linux distros, and ubuntu. The reason why companies like twitter, tumblr and facebook succeeded where many other startups fail.

This is important for AO3, because to succeed in its mission to be a home for fanwork to preserve it, it needs to be a hub. Right now most if not all of the alternatives of the AO3, major fandom archives, even those run by companies, are around the same caliber of product and service quality. Thus AO3 has no difficulty attracting users. However when next version of Fanlib comes around trying to monetize the fandom with no concern of their well being, will we be so lucky or will it come in form of a product of the polish level of pinterest or tumblr? How will then AO3 catch to be a viable alternative to the fans, since we do know average user do prefer usability over privacy and social concerns. It is for this reason if nothing else that AO3 needs aim for better.

If someone, a commercial someone comes in with a product, that might not care about fan interests but still offers a better product, people will migrate. Thus it is in AO3's best interest to be competitive.

Date: 2012-08-14 09:44 pm (UTC)From: [personal profile] ira_gladkova
Thank you for discussing this.

I've actually asked about doing something nearly identical -- in late 2011 and early 2012, I and some other people were talking about these issues and asking about similar ideas, and in January 2012 I laid it out explicitly within Board. With a committee this size, managing the volunteers, managing the project in terms of coding and growth, and making policy decisions on the project is simply too much to ask. I talked back then about splitting AD&T into design, coding, and testing teams that would be used across all org projects, with probably an AO3 governing committee as well. The idea was rejected -- at the time, I did not handle the proposal well, and its scope was not wide enough.

Now the Board conversation about org structure has started again (as you noted and linked) with a broader scope. I'm hoping to talk about that in a post of my own soon, so this is very much relevant to my interests!

I'm not certain about splitting coders into two committees, but your arguments in that direction are compelling. This might come in part because I'm from a sort of crossover background, in that I've done both back and front, so for someone like me I would love a centralized place for all coders. I also find it easier to pass work back and forth, especially on asynchronous web apps. Do you think some kind of two-tier structure would work? A coding committee with front and back subcommittees? Or maybe a coding committee that manages feature development across both teams and maintaining consistent development standards, presiding over front and back end committees that are more focused on volunteer management, community-building, knowledge-sharing, etc. within the expertise groups?

Overall, I clearly agree that separate and org-wide committees for design, coding, and QA are needed, as well as a governing committee for the AO3. The current setup asks too much of everyone and creates too many conflicts of interest, and undermines the potential of the AO3.

Date: 2012-08-15 03:53 pm (UTC)From: [personal profile] highlander_ii
highlander_ii: Jarod from the Pretender in sunglasses licking his lips ([Jarod] tongue and shades)
Something else that might be helpful for the OTW w/ regards to coding is have a formal 'request' form/process in place to submit feature requests that require coding to the appropriate committee for approval and addition to the queue. Right now, I think, chairs email AD&T, then AD&T does a yea/nay on it.

Date: 2012-08-21 02:00 pm (UTC)From: [personal profile] highlander_ii
highlander_ii: Chris Pine kneeling on the floor holding a camera to his face ([Harvey] profile)
I was thinking more of an 'internal' process, rather than going through Support. And not for every little thing, but more for [Committee] needs X -> [Committee] fills out Form -> ADT gets form.

B/c right now, I think the process is [Chair] emails ADT, which is okay, but isn't much of a formal tracking process.

Date: 2012-08-21 02:00 pm (UTC)From: [personal profile] highlander_ii
highlander_ii: Jarod from the Pretender in sunglasses licking his lips ([Jarod] tongue and shades)
=)

this is currently the only Jarod icon I have on this account... should change that at some point. =)

Date: 2012-08-15 05:48 am (UTC)From: [personal profile] highlander_ii
highlander_ii: Alex Cabot sitting on a bench with Oliva Benson ([Alex] sitting with Liv)
I initially joined up as a 'QA tester' and, in the beginning, the 'pound on things until they make noise' idea was fine - that 'experiment' stage - but later, we really *really* need a more concrete 'test plan' - both generic and feature-specific - and didn't have one. There was a plan, but it was a bit too generic. It's one of the reasons I cut back on the testing I was doing (one of several). I haven't done a lot of testing since then. I don't know if this has changed since I slid away from testing.

I'm also of the opinion that if [you] code something [you] shouldn't then turn around and test it, b/c [you] won't be able to see certain mistakes that another set of eyes will pick up on pretty quickly. (( generic 'you' in all these here ))

All-in-all - I agree with what you have here. I fully agree that the prioritization of feature implementation should be outside the hands of the ones who will do the implementation.

Date: 2012-08-21 01:14 pm (UTC)From: [personal profile] tiyire
tiyire: (Default)
These are excellent ideas. Clarifying the different roles of AD&T seems absolutely necessary (from an outsider, i.e. Wiki staffer perspective), and separate teams for coders/designers/QA sounds like it would be much easier for e.g. Wiki to borrow some expertise, which would be fantastic.
However, I'm concerned that more committees require more chairs and, more importantly, even more and better additional communication than the org urgently needs anyway. At the moment I don't see enough work being done to support and teach the chairs and to establish better communication within the org, and I think without that even more committees might make that so complicated that the hoped for benefits might be cancelled out if not worse.

Date: 2012-08-21 08:35 pm (UTC)From: [personal profile] tiyire
tiyire: (Default)
Thanks for the clarifications!

I didn't mean that the communication problem would make your suggested changes impractical in general, just that I think we need to make sure that we tackle this problem first or at the very least at the same time. I know that Volcom and Board are working on some things in this direction, which is great, but I haven't seen much progress so far - which is not a big surprise, to plan these things well takes time, but I don't know how long it will take and how fast we can expect things to improve. Yes, we can use our experienced managers better and we can train our chairs better and build our own managers, but it will take time.

I don't think your proposal would lead to more communication troubles overall if we have figured out a better way to communicate in general, in committees and between committees. I don't know exactly how much of an issue this currently is in other committees, but we recently had discussions about this in Wiki and I would be very surprised if we were the only ones for whom this can be tricky. It would probably lead to more people being members of more than one committee, but many seem to do that already and I don't think it's a big problem.

I agree that this split would make things clearer for a lot of other committees as well. I'm not sure if it would make things easier or harder for us in Wiki - it probably depends on if we have our own tech person (which we now have, yay!) We need something of everything. But it would definitely be much easier on Systems ;)

Sure, just drop aethel a note and I'm sure she'll arrange something.

Date: 2012-08-26 05:17 pm (UTC)From: [personal profile] anathomical
For context: I'm currently a member of AD&T, and have been for approximately 18 months. That said, I'm speaking for myself in this case, rather than in any sort of official capacity. In my day-job I work as a coder and scalability specialist for web-sites, and do a small bit of side consulting on organization and corporate culture construction.

First of all, thanks for bringing this up. There are quite a few organizational issues that you correctly identify. Especially with regards to prioritization and supporting non-AO3 projects, and those definitely need to be dealt with.

(I had a section here analyzing various approaches to developing software, but realized it wasn't my main argument, so cut it.)

I actually think that as an organization the OTW would have significant problems implementing a full-on version of either Waterfall or Agile methods, and a lot of those problems boil down to variations on directive authority.

I basically think that the problem you're trying to solve is one of how to get coders to work on the things that need working on. Translation interfaces and unexciting features and bug fixes and non-AO3 code and all the other things that sort of languish with insufficient attention at the moment. This basically boils down to a directive authority problem.

Basically: in order to direct coders (or anyone, really) to work on a specific feature or idea, you have to convince them that it's worth their time to do so, and you have to do that in the face of competing things that they could be doing with their time instead. So if you want someone to work on (say) translation interfaces, you must convince them that doing so is a better use of their time than (say) reading fic.

The vast majority of corporations solve the problem of directive authority by paying their people. That creates a contractual (both legal and social) obligation to do the work that the corporation specifies rather than (say) reading fic. Unfortunately, at least at the moment, the OTW doesn't really have this option, so we have to fall back on one of the other ways of generating directive authority.

Other than self direction, which provides directive authority purely internally, almost all other forms are entirely social. It is, in fact, almost exactly like trying to get someone to write a specific piece of fic that you want to read: they might do it because they think the idea you have is awesome, they might do it because they're your friend, they might do it because they're getting something in exchange, they might do it because they want to impress you... Well, hopefully you get the idea: there are plenty of reasons they might do it, but none of them are just because you tell them to.

All of which means, I suspect, that the reorganization you're proposing won't actually solve the problem. Because what it sounds like you want is the ability to have higher-ups agree on the importance of a feature, and then to have the ability to make that feature happen with what amounts to managerial buy-in. And, let's be honest, that sort of process tends to be both more coherent (allowing for a sort of shared vision for direction) and more efficient (in that things that need doing get done).

However, because of the nature of the organization, I am pretty sure that you quite literally can't get this kind of structure. Which means that for a given feature to happen, or a given bug to get fixed, you either have to be a coder, or you have to campaign to convince a coder that they should spend their time on it.

They might do it because they think the idea you have is awesome, they might do it because they're your friend, they might do it because they're getting something in exchange, they might do it because they want to impress you...

But they won't do it because you tell them to. And I literally can't imagine that reorganizing would change that.

Which isn't to say that AD&T's current structure is perfect, or even optimal. I think there do need to be some changes made. But I think that if your goal is to have more directive authority over the activities of the organization's coding volunteers, this is the wrong way to go about it. I think that we should actually be having two separate conversations.

1) A conversation about project management inefficiency within the AD&T committee. This includes things like the fact that Google Code is flat-out awful for managing large feature requests along with other procedural and tool-based issues.

2) How do we we provide ways for people to get the work that needs to be done for the org to move forward done? If you can't write the code yourself, how do you get someone help? Current experience demonstrates that simply getting buy-in at the committee level is largely ineffective as I doubt that anyone's going to argue that (say) translation interfaces aren't important, but that work still hasn't been done. I propose that, additionally, proposals along the lines of yours would also be largely ineffective, and for similar reasons. Which means we need some other method for doing what is clearly an incredibly important task.

That is the conversation I think we should really be having. Not (to go back to my metaphor) "How do I create a structure that makes people write the fic I want to read", and rather "How do I create a structure that makes it as easy as possible for me to convince people to write the fic I want to read".

Directive authority is hard.

Yours,
Ana

Date: 2012-08-28 12:46 am (UTC)From: [personal profile] anathomical
First of all, I realized after I posted my comment that it might sound like I was dismissing your points about organization, which was not my intention, but then I figured I'd let you respond before tossing in more thoughts! So, sorry if that's how it seemed.

Because you are clearly talking about more than directive authority. BUT I think that structure imposed from outside makes it harder to hold onto directive authority, so I do think that part of how this whole thing is presented is worth being careful with. Especially since ADT has been discussing some of these problems internally. Not that we don't need outside input!

Also, I do want to clarify. My point isn't that coders work on what they like, it's that they work on what they are motivated to do. In fact, most of the bugs I end up fixing aren't ones I run into, but are ones that people I know run into. I fix things for my friends. I might also fix things for recognition, either external or internal to the committee. I don't fix things just because someone tells me to (though I might fix something that someone tells me to if I think it will result in recognition, etc.). To use my by-now stretched fic metaphor: I might write a fic that I wouldn't otherwise because I signed up for an exchange, and that's part of the responsibility. But if I don't have fun, I probably won't sign up for the next one. Anyway, while this is all worth discussing, and really has to be dealt with for any sort of organizational changes, let's dig into the actual things you want to do a bit.

Now I kind of regret cutting my discussion of Agile processes because it sounds like you've either had bad experiences with them or just not been around them much, but they're generally considered: (a) much better at rapid iteration, (b) much better for user-focus (a knock-on effect of rapid iteration), (c) harder to scale (in that middle-management of coder teams have to be coders, not project managers, for maximum efficiency). With the exception of that third point, which can be mitigated, agile development is significantly more in line with our goals, I think.

Furthermore, while Agile doesn't scale as cleanly, we are nowhere near the limits of that scaling. I work at a 2-million-ish-user startup at the moment, and not only do we use Agile, but I'm pretty sure that we could double the size of our tech team, maybe even triple it, before even beginning to run into organizational issues.

I suspect that it's worth diving into more detail on specifics, but deciding on Agile or Waterfall doesn't really address what I take to be some of your early concerns.

I will say that it's interesting that you seem to take almost the exact opposite approach as me to the problems I think you're focusing on. One of the issues we have (and I think ADT is aware of this) is that Coders are too dominant in decision-making and priority setting. Testing is especially pushed out of a lot of those conversations, and the results are unfortunate. Additionally, we're still not iterative enough, and our release cycles are far too long.

I read you as suggesting that the solution there is to draw clearer distinctions between various groups so that each sub-division of ADT ends up with its own strong voice in its own area, and then use a coordinating committee to increase and encourage cross-group communication. I tend to take the exact opposite approach. If a group within ADT has less influence/input than they should, the solution is integration rather than separation + more management.

I feel like my points are all over the map, not really managing to come out all that coherent. It's been one of those days.

I hope it makes some sense,
Ana

Date: 2012-08-31 03:33 pm (UTC)From: [personal profile] anathomical
I just wanted to give you a quick heads-up that I'm at Dragon*Con this weekend, so there's a pretty good chance that I won't manage a real response to this until Tuesday or Wednesday after I've recovered a bit. But I will be thinking about what you've said here and working on formulating a response.

Just didn't want to disappear without warning!
Ana

Date: 2012-10-08 04:31 pm (UTC)From: [personal profile] anathomical
Ugh. I've owed you this response for, like, a month. Sorry for the delay! It's been a busy one...

ANYWAY!

I think your point about work duplication is a good one, and that's something that probably should be addressed. I don't know of a good alternative solution myself, but I figured I'd mention that I see the point your making.

I think I see what you're getting at, but I also think that you're conflating separation with proper procedures. Having separation is no more or less likely to result in good, well-documented, followed procedures that integration. And lack of good, well-documented, followed procedures are a problem. A serious one, I think.

So I'm all in favor of addressing the procedures issue, but I'm not sure that separation is the best road forward. Here's my basic thinking:

Separation allows for some small subset of people to do all the mental overhead in requirements collection and procedure generation. That is, whoever's in the top level committee becomes responsible for talking to all sub-groups and figuring out how they operate and what they need, and then generating a set of procedures for that group. The advantages here are that 1) The group is small, which can lead to higher efficiency per-person, and 2) the procedures are formed by the same small group of people, so theoretically those procedures should mesh cleanly with each other.

The disadvantages are that 1) the group requires a preliminary procedure-generation phase in order to generate the procedures that it will use to generate follow-on procedures, 2) the extra procedure-step means that changing procedures is hard, reducing flexibility (which, in turn, increases pressure to "get things right" the first time, and can make it difficult to adapt when something unexpected happens).

I'll admit I've got a bias toward agility, so the reduced flexibility of a stricter approach is a huge downside from where I sit, but I think that actually lines up with the AO3 as a project. What needs doing is often changing, priorities shift, and new features grow organically; and being able to respond to those changes as quickly as possible is a good thing. So I'm generally resistant to a more complex, baroque organizational structure.

That said, a more agile approach is not incompatible with more product management. People to scope out new work, help with prioritization, make sure that good QA requirements are in place so that we know how to properly test something... all of that is important, and largely isn't being done very well at the moment. So figuring out how to bring in and use some product managers would probably be a good idea. Ditto on design (we seem to have a really hard time getting designers and UX people in... not sure why...)

Rather than making the organizational structure of the committee more complex, I'd be more inclined to do two things: 1) put together a short-term subcommittee tasked with figuring out what the pain points of the various branches of ADT are, and coming up with some organizational solutions. 2) Look into actively recruiting one or two non-technical people to help with product management (they need to specifically not be engineers so that they can't get sucked into actually writing code).

I think that lets us begin moving forward on figuring out and implementing better procedures, and lets us add some more organization to the whole process, without the added overhead of more Organization. Basically: I think you've identified a number of issues with the way we do things, but I also think that most of the solutions can be achieved without making things more complex.

But, as I said at the beginning, this doesn't necessarily solve the issue of duplication regarding things like front-end coding...

Sorry again for the delay,
Ana

Date: 2012-10-09 07:25 pm (UTC)From: [personal profile] jennyst
jennyst: Jenny on a photo of space (Default)
Sorry to butt in here, but part of the issue with getting product managers, designers and UX experts is the current structure and lack of procedures. Eylul is an experienced designer and UX person, as well as a computer science graduate and ex-QA lead for OTW. Many, many experienced project and product managers have taken one look at our current or past processes and run in the opposite direction. Non-technical people have arrived and have been pushed away.

And the thing with moving from pain points to solutions will hopefully be done in an organised way by the Strategic Planning committee on a larger scale, but some of it needs to be done by individual committees now, without waiting for that.

I don't think Eylul's proposed structure is quite right, but the current situation is clearly not enough, and we have some good procedures that aren't being followed, so that can't be the whole answer.

Date: 2012-10-10 03:01 am (UTC)From: [personal profile] ira_gladkova
Like [personal profile] jennyst, I apologize for butting in, but want to second her points.

In addition, I want to emphasize that Eylul's plan does not make the structure of the committee more complex -- because there would be no single committee to do this to. AD&T as an entity, or its equivalent, would not exist. The idea is to make several much smaller and much simpler committees, because right now AD&T is *already* too complicated with its huge size and multiple subcommittees. Adding more subcommittees on top of that exactly *is* making the committee more complex. Your point about product management is also addressed in Eylul's proposal, because that's the whole point of an AO3 governing committee -- as far as the AO3 product goes. Fanlore is also a product, and other projects also have tech needs -- these would each be managed individually.

This leads me to another note. One of Eylul's biggest points is that other areas of the org besides the AO3 need design, coding, and testing, and I feel like this is sort of getting lost in a lot of the responses, as the focus continues and continues to be on the AO3. I know that this is the project that many people are most interested in, but as an *org* the OTW needs to ensure that resources are distributed to *all* our projects. Right now, they're not, and one of the reasons for this is that the committee that is de facto managing the AO3 project is also the committee that does all the coding. Of course the AO3 will be prioritized in this case! This is an area where I absolutely see no solution besides forming an independent and separate AO3 governing committee.

But that's just one of the proposed splits. As far as splitting design, coding, and testing, I wanted to address two things you said. One is that we have trouble getting experienced UX people. Another is that the split requires a procedure-generation phase. I strongly believe that we can't get experienced UX people *because* our structure and procedures are so unusual and -- to be frank -- ultimately non-functional. We get the code out, eventually, but at an entirely excessive cost in terms of burnout, ignored voices, delay in non-AO3 projects, and sometimes even at the cost of code quality itself (especially when testers don't get heard enough or are not given as much time). A procedure-generating phase seems required in any case, because the current procedures -- or the way they're followed or not-followed -- do not work. I'm speaking not just taking into account the final product of the code and design, but in terms of a holistic understanding of the health of the org: how our people are doing, and how *all* our projects are doing.

There are points in Eylul's proposal I disagree with, and many areas that would need much more elaboration (e.g. more on release management). But I do think she captures a lot of the issues. As Jenny said, we do have a lot of good procedures that aren't being followed, and I'm most interested into getting to the bottom of why that is, and building solutions from there.

Profile

eylul: (Default)
eylul

October 2012

S M T W T F S
 12 3456
78910111213
14 151617181920
21222324252627
28293031   

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Oct. 18th, 2017 06:09 pm
Powered by Dreamwidth Studios