* Embedded Development:
  The Role of an Involved Community in Social Software Design

Delivered by Matt Webb at the Interface and interaction: Social software event
on 9 December 2004, part of the Designing Design seminar series at InterMedia
(University of Oslo).

Full video from the event is online:
http://www.intermedia.uio.no/projects/designingdesign/social_software.html

The presentation and this full talk can be found at:
http://interconnected.org/notes/2004/12/EmbeddedDevelopment/

Under discussion is background to the development of the Evnt project:
http://p.evnt.org/blog/ -- project weblog and background
http://b.evnt.org/ -- current beta.


The text of the talk follows. Each section corresponds to a slide.

#

It's worth looking very quickly at some different models used for software
development, before we get into this, and I'll take two extremes. I'll cover a
bit of history of development so you can see where this embedded development
model is coming from.

#

The first is the traditional waterfall model. In this you start with
requirements gathering with the client. You write down all the different flows
of data, people who are going to be using the system, and completely define it.
It then goes to designers to decide what it looks like, then to programmers who
implement it. Or it goes the other way: Programmers implement how the database
and the ways of changing the data work, and designers then make it look pretty
at the end.

The second is the opensource model. Somebody releases a utility or code library,
and many people make use of it. They feed back their comments, sometimes by
email, sometimes by changing the code, and the small program gradually turns
into a big one, with each incremental feature addressing a need. Code is
sometimes rewritten extensively to add new features.

Both of these models depend in some way on *requirements*. The first is about
deciding upfront what the requirements are. The second is the software is built,
in a series of steps, to fulfil a requirement. If the software is used for
anything else, beyond what those original requirements anticipated, that's a
complete surprise.

#

At this point, we enter recent history. We noticed that, regardless of the
requirements of the software, there are *always* unintended consequences. I'll
give you a very simple example from the weblog world. Blogging software started
to let people keep webpages of links and commentary. Unintended consequence: The
ease of updating the webpages meant that they could be updated quick enough to
use them for conversations. So the weblog pattern incorporated that, and weblogs
began to be archived with special links to point to each salvo in the
conversation, called permalinks. Unintended consequence: Now weblogs could be
divided up into posts, and given structure, they could be divorced from the
webpage into post-size units that can flow, be added together, filtered and so
on, and we call this RSS.

Anyway, what we learnt from unintended consequences - the fact that *everything*
has intended consequences - is that we never know the requirements up front, and
our response to this we called adaptive design.

#

Adaptive design said that the important thing was not meeting the requirements
alone, it was also the properties of what you created. It realised that the
design of a product, be it software, or an interface, or a car or an
ecosystem--it realised that this design was a journey which couldn't be entirely
anticipated at the beginning.

I'm using the word design here to mean the discovery of requirements by a
process of making.

Adaptive design means different things in different contexts, and there are
people way more articulate than me in discussing it, so I'll keep to what I
know, which is software that works on the internet, and specifically social
software.

As I read the dialogue, when it comes to software we talk about adaptive design
in two main senses:

- For the interface, we say that the user must be able to twist the software to
their own devices. We have to enable the unintended consequences of the software
by not making too many assumptions. For example, the windows and icons desktop
interface is so adaptive we don't really think of it any more. It has a starting
structure, sure, with places for your documents and pictures. But it's pretty
liberal after that, and it doesn't dictate a particular project-based working
pattern, for example. We can observe how the software is being used and bake
this into the code, or see how people want to use it and bake that in too.

- For the backend, the software itself, there's a type of adaptibility needed
for the developers. Given we don't know the requirements upfront, the structure
of the software that meets the initial requirements is really important. It has
to be able to grow. It has to be able to change. A developer must be able to
move through the software and fix a bug in a hurry without damaging the overall
structure.

And it has to allow change in the right places too.

There's a midpoint between the backend and the interface, and that's how the
software itself can be extended with other pieces of software. Users are free to
install their own software on a desktop interface. But most users don't write
that software themselves. This means we have a bunch of people who are writing
and distributing software, taking advantage of the hooks to datastructures and
so on we've provided in the code, and a bunch of people who are making use of
this software to adapt their own experience of it. There are communities here.

#

Okay, let's leave this for a moment. I want to talk about social software.

Parallel to the discussions about adaptive design were the discussions on social
software. What social software acknowledges is that people don't treat software
as a closed universe where the only information they have is information the
software provides, and the only options they have are the ones the software puts
infront of them. They're people, and people have drives.

People like to protect their reputations. They like to save face. They don't
like to put other people into situations which are uncomfortable. They like to
share: photos, ideas, links, conversation. They like to form groups.

To do these they need to recognise each other, there needs to be some form of
identity. There needs to be consistency and risk, because if they don't risk
their good names then they won't treat other people well. And so on, and so
forth. Ultimately social software recognises that, without constraints, people
are self-interested and rude.

This is what social software understands, and so we build software that lets
people express these drives and these social calculations online. That's
software like Friendster.

Next, we build software that lets people find individual utility - they can
individually store their bookmarks, say - and then the social aspects of the
software sit around that. We then start getting group utility out of the
software, and that means that there are new requirements for the group too. The
group needs to be able to support itself. It takes on its own existence and
needs to be around even when some of the members aren't, so it needs a
representation in the interface, some kind of visibility, and so on. There's a
community use of the software.

I'll call this "side-effect social software." The best properties of the
software come out from group behaviour, and the group behaviour is a side-effect
of people operating in their own best interests.

The problem is, we're still figuring social software out. I couldn't, for
example, go out right now and build something that would work perfectly for some
arbitrary group of people, I just don't know how. One piece of software that is
heavily utilised - email - is absolutely caustic to groups. It poisons them. It
defaults to one-to-one communication, so group communication falls apart. You
can't tell if people are talking behind your back. Conversations dissolve into
flamewars. I regard that as a design problem, but it's not something we really
know how to fix yet.

This is where social software and adaptive design meet. We don't know how to
build good social software because we don't sufficiently understand the human
and group factors--I don't think we'll have more than rules of thumb in the near
future. We need to continually discover the requirements, and adapt as we go.

#

It's now I can start talking about, specifically, my project in London.

We're building a piece of social software for people in the media arts sector,
centred on London. The software will help them track the conversations going on
around events they attend, like exhibitions and workshops, publicise their own
work, and discover other work similar to their own. It'll be useful for both the
people who participate  and organise the workshops, and the people who just like
to view shows.

We've been asked to build this because the people who make the video
installations, the people who run the networked computers and the online art,
and put on the conferences--they tend not to know each other, and so they don't
get the benefits that shared effort and shared experience would bring.

The thing is, the difficulty is, media arts is a technical world, and a large
area of it is based in the semantic web, and also in social software. So why
aren't they doing this kind of work already? Social networking should be prime
territory for them. There are two answers:

- there's no call for this kind of work yet. Nobody really realises that this
kind of visualiser of the community is missing. People have been experimenting
with systems like it, but nothing that's really taken off.
- this kind of work isn't possible because the infrastructure doesn't exist yet.
We want to map data about media arts events, but the data isn't in a machine
readable format yet.

A specific objective of the Evnt project becomes: foster an ecosystem where this
kind of project is possible.

Another objective is: Make this project run automatically. Once we've all moved
on to different jobs, this system needs to keep running--events need to go into
it, conversations need to be tracked, people need to be able to view their
calendars.

This is ecosystem design. We're now not just trying to build a system which
meets certain requirements. We're trying to build a variety of loops which drive
themselves along. Perpetual motion machines of use and metadata.

#

To summarise: We have a single system. It's used for tracking event
conversations. It's side-effect social software, which means that it has
individual utility, and out of that comes some benefit for the community that
uses it. In this case the benefit is that there's greater visibility of work and
conversations, both to people within the media arts sector, and people on the
outside.

This single system needs to be populated with data which doesn't exist yet.
After money to run the project is gone, it still needs to be populated. The
people who put this data in are members of the community, because they're the
ones who know what's happening.

The third point is: We don't know how to build this system. First, because it's
social software, and we don't know the requirements, ahead of time, for social
software, not in detail. Second, because we don't entirely know the type of
events and conversations the community will want to use the system for.

#

It's at this point I return to the software development models from earlier. The
waterfall model isn't appropriate here because we don't know the requirements.
In theory we could just decide completely what we want to build and put it out
in the real world, and then it'll sink or swim according to whether the
community adopt it or not. But we want to maximise our chances of creating
something that'll be used, so we can't do that.

We also can't go the open source route as a development methodology. Open source
relies on many people having copies of the system and adjusting it as they go
along. By its very nature, a system which is for a single community must only
have one instance.

What we can do is create a system that is able to grow and adapt, in all its
parts. This means that we decide on a minimal set of requirements to begin with
and then let it grow.

We have a five part plan to do this, which I'll run through now. Remember that
at each point, the system must be a living thing. People must be able to move
through it and get utility at every single stage. If they can't do this, we
won't have people using the system, and if there aren't groups of people using
the system, we won't know what to do next.

1 - First usable system

At the first point, users can do almost nothing, but it's an almost nothing
which is still useful. They can go into the system, put in an event to track,
and get information from around the web back about it.

#

2 - First round technical community engagement

The first round is about establishing a circuit a user community can trace a
path around. This second step is about establishing a circuit which binds our
system into the greater technical world. Remember, we're about building an
ecosystem here.

Circles are like life. They're loops which create themselves as they go along,
like the metabolic cycle which converts chemical to chemical to chemical. It's
powered by sunlight, and is an engine to extract energy and create itself in the
next generation. The metabolic cycle doesn't exist in any single body, it exists
across organisms. There are organs we call plants, and organs we call animals,
all different organs which chain together to complete the circuit. While the
circuit pumps away, it leaves behind remnants, and we call these remnants flesh,
or people. We're side-effects of the self-creating circuit of the metabolic
cycle.

Part of ecosystem design is to create these self-sustaining circuits, and the
only way they'll live on after this project is if they're taken on by the
community itself. That's why we need to plug into the community. In a way, the
energy of the community circuit passing through this project is what brings it
to life. It leaves behind the events metadata we work with.

When I talk about circuits, then, I'm talking about flows of data that persist
without my involvement. I'm talking about an incentive loop: Performing some act
which creates something that makes itself more likely to occur again.

The question is, why should the technical community be involved in this circuit?
What's in it for them? When the loop is established, it's obvious: They'll want
to consume data because of what they get out of it. We then use the produce of
that data, feed it to our users who create something else, and then we feed that
back out to the community. The difficulty is how to start the loop. What
strategy do you use to engage a community in something which is really there
just to help you, although it'll benefit them eventually?

Think about it like a conversation. A conversation is a loop that persists. It's
hard to leave a conversation, if you're doing it right. But how does it start?
What's the opening salvo? For people using the system, for non-technical people,
you give them something they can use. It's a gift.

And that's how we have to think about the technical community aswell. Technical
people communicate in code. Although regular words have value, in that they're
talking about ideas, ideas are cheap. To speak words you needn't invest any more
time than the time it takes to say them. But to speak code, you need to invest
the time it takes to create that code. Making is hard work.

If you're going to ask a technical community to invest time in creating code to
be part of your system, you have to give them something of value to start. You
have to give something in the gift economy to demonstrate your willingness to
participate.

This is the real strength of free software, free software as opposed to just
open source: You give away code and the control of the code. You say to the
hackers: You can take my code at any time and do what you will with it, you can
fork the project. That's the gift.

In our case, the gift is control of how we talk about events, and the events
data we gather. A standard for events data will benefit the community because
they can build other projects with it, and it'll sae them time and effort. The
events data is something they can make use of. So we have to talk in objects
instead of words, and this is our gift to open the conversation.

You can see this pattern in action watching Apple engage with the open source
community. Apple's browser, Safari, is based on an open source effort called
KHTML. Apple reaped the benefits of their hard work by taking the code and
adding to it, creating their own, new browser. They took the code and developed
in secret for several months. However, they knew that had to have an ongoing
relationship with the developer community, and on the day of the big
announcement they emailed the developers mailing list and they did two things:
They introduced their technical developers, so it was a peer relationship. They
also gave back all the changed code, all the improvements they'd made, at the
very same time.

I know this is mutually beneficial for both sides. Apple don't get the benefit
of open source unless the code they have is being continually updated by the
community aswell as themselves, but the timing is significant: The introduction
to the community, the way the conversation was opened, was not really with words
- not with a negotiation of terms of engagements - but with a meaningful
quantity of work, a gift in the gift economy.

As the users and technical communities move through our system on the circuits,
so we adapt, and this is step 3.

#

3 - Iterative development

The basic loop now formed, it's time to add complexity to it. You can see this
pattern in action at the photo sharing site, Flickr. The basic site they
launched was very simple: You uploaded your photos and showed them to your
friends, and you kept your friends in a list. This was the basic loop. Now a
community was on a circuit through their system, they embellished, adding
different levels of friends, groups, privacy and so on. They added features.

They also completely transformed the way the site worked, moving from a
full-screen Flash application to a system of webpages--the interface completely
changed, but the circuit remained, and the circuit grew in completely.

Flickr also added ways for technical developers to get involved. They opened up
their system as a way to store photos and look after user security from any
other system. That was their gift. In return, developers have built applications
on top of Flickr, creating a circuit from Flickr into the wider technical
community and back again. They're creating an ecosystem.

All the time, Flickr are adjusting how their loops work. Their systems are
adaptive, and they change such fundamental things as the date assigned to
uploaded photographs. In a traditional software development model, that's the
kind of thing you fix at the requirements change and never change. And there's
no real distinction between the technical and the user community. Like the
desktop interface, the non-technical users take on the applications made by the
technical people, and, if popular, these are supported better by Flickr itself.

This is the iterative approach we'll be following. You continue giving gifts to
the community, and they continue giving gifts to you. You adjust your system to
make it more appealing to the community--to make it work faster for the uses
they need, to gather more data where they need it, and to provide more nuance
where it's needed.

#

To complete these five steps, step 4 is to involve more users. Once the system
has been iterated and expanded, it's time to involve people who are very
different from the core users. They people need more to persuade them to use the
sytsem as a whole, as so they'll only come on board when there's an ecosystem of
applications to use.

For us, these are the media artists are currently aren't really communicating
with one another. It would be unrealistic to think that we, a single project,
could engage all corners of this hard to reach community. But it's the intention
of having a constellation of developers, a community of developers, making many
pieces of software that work together, it's the intention there to engage the
user community very widely.

The fifth step is the final step: We launch fully, let the system go, and hope
the various loops we've set up will allow it to persist.

#

What I've described is something I'm only really able to describe as embedded
development. The decisions that are made in the way the software is written and
what it's good for are made not just by me and the rest of the team, but by the
community as a whole. If they don't have influence over the project then they
won't stick around, and they won't make use the various standards and formats in
their own projects and software.

It doesn't follow the waterfall model, that's clear. But it also doesn't follow
the open source model. The open source model is still very individualist. Each
person developing the software adjusts it according to their needs. In both
models the individual has control.

In this model, as a developer, I *don't* have control. I only decide what to
tackle next.

The model that Flickr follows, and we're trying to, does for software
development what social software is doing for design. Instead of developing
software for individuals, it's attempting to use the opinions of a community to
develop software that exists to *support* that community, and to foster an
ecosystem.

And therefore I'd describe my role in this project as a "developer gardener," to
weed out what's bad and to cultivate what's good, but ultimately to grow my
software and its community in symbiosis.