tools-yak
[Top] [All Lists]

[tools-yak@collab] Re: H2O evaluation (fwd)

To: <tools-yak@xxxxxxxxxxxxxxxxxxx>
From: <cdent@xxxxxxxxxxxx>
Date: Thu, 27 Feb 2003 14:00:54 -0500 (EST)
Message-id: <Pine.LNX.4.30.0302271333420.5886-100000@hot.burningchrome.com>
On Thu, 27 Feb 2003, Eugene Eric Kim wrote:
> On Mon, Feb 24, 2003 at 10:11:40PM -0500, cdent@blueoxen.org wrote:
> >  - we didn't go into the discussion with a shared understanding
> >    of the question being asked
>
> And I would argue that H2O was not helpful in facilitating that shared
> understanding.  I responded to two posts asking for clarification and
> further discussion, and yet the original authors could not respond,
> because H2O wouldn't let them.    (01)

That doesn't necessarily argue against the constrained
environment in H2O but instead might suggest that some out of
band conversational medium might be helpful, like the
back-channel stuff mentioned by Dennis:    (02)

  http://collab.blueoxen.net/forums/tools-yak/2003-02/msg00093.html#nid08    (03)

> In the interest of being as objective as possible when doing tool
> evaluation and comparison, "motivation" has to be one of your
> controls.    (04)

Totally disagree with this, but they you probably know that.
Some points:    (05)

 - Objectivity in tool evaluation is a myth. Tool "usefulness"
   (whatever that means) is entirely too context and user
   dependent to make any real commitment to objectivity. I think
   perhaps that was part of Earth's point when he was suggesting
   use cases:    (06)

     http://h2o.law.harvard.edu/viewThread.do?postId=2101#2101    (07)

   "It depends" remains the response to "is this any good." It
   depends on what you are doing with it.    (08)

 - Motivation is really the only thing that matters--you
   basically said it yourself    (09)

   http://collab.blueoxen.net/forums/tools-yak/2003-02/msg00112.html#nid08    (010)

   --but it too is context and user dependent. Like I said
   motivation comes from the shared understanding and goals that
   result from _real_ and shared need.    (011)

So, basically, what I'm arguing is that there is not a _real_
investment in the use, testing and development of H2O, or any
other "collaborative" tool for that matter. There is engagement
is those issues, but that's not quite the same thing as real,
motivated, desire or need.    (012)

Most of us are moderately disgusted with the incapabilities of
our email clients and our browsers, but using them is pretty much
how we spend the whole day. Huh.    (013)

> >  - the rotisserie settings were wrong for the type of discussion
> >    we were trying to have:
> >    - 24 hours was too short for people who have other things they
> >      need to be doing
> >    - 4 rounds was just enough to get us started
>
> Agreed.
>
> Who controls these parameters?  Were there guidelines for determining
> what these parameters should be?    (014)

The person who creates the rotisserie makes those settings. I
asked for input beforehand:    (015)

  http://collab.blueoxen.net/forums/tools-yak/2003-02/msg00063.html#nid013    (016)

The defaults were 24 hours and 2 rounds. I assume those are for a
classroom setting where you want each student to produce once and
respond once. I went for 4 because I wanted a little more
discussion but I didn't want things to go on so long that we ran
out of steam.    (017)

> >  - while we don't have any apparent winnowing of discussion in
> >    the threads we produced we do have three postings that
> >    received high evaluations (where high is defined as greater
> >    than the maximum points one reviewer could apply):
>
> Several gripes here.  My first is starting to become mantra for me: In
> order for community rating systems to have any value, there must be
> some shared understanding about those metrics.  A rating of two has to
> mean about the same for me as for you.    (018)

I disagree. I think in an actual group (one where the people are
committed to developing a shared understanding and have some kind
of shared focus) a positive rating has value simply as the
following statement: I, your peer who you have chosen to trust
enough to be in a group with, think you should read this one. The
bigger the number, the more your peers think you should read it
and care about it.    (019)

> Also, I believed that ratings determined what threads would continue.
> In a small group, one person could easily throw everyone off a thread,
> even if everyone else in the group believes it's important.    (020)

The docs for H2O say that the ratings are designed for use after
the discussion is done so the leader and the rest of the
participants can have some support in picking out which of a
large number of threads and postings should be used as starting
points for further thought.    (021)

-- 
Chris Dent
cdent@blueoxen.org    (022)


-- 
This message is archived at:    (023)

http://collab.blueoxen.net/forums/cgi-bin/mesg.cgi?a=tools-yak&i=Pine.LNX.4.30.0302271333420.5886-100000@hot.burningchrome.com    (024)
<Prev in Thread] Current Thread [Next in Thread>