Amy Dolzine from accounting firm EY recently reflected on five years of observing Yammer groups, both public and private, and like many of us has been promoting transparency and open group collaboration. However, she made the following observation:

“In my observations, completely without data, it feels like having private spaces where the audience is limited to a select group, ironically enables people to be more open. When you ask people in an org to be vulnerable to all, people are not as yet willing to ask the difficult questions, and people are less willing to answer. But in a smaller subset they are.”

Amy’s post intrigued us, perhaps because in some way it resonates with what we have observed ourselves. We often hear that private groups provide a safe place to speak openly. At SWOOP we are the ‘data people’ and because of that we felt compelled to see what evidence, using our Yammer benchmarking facilities, we could find.

Firstly, we had already conducted an analysis of “Open” vs “Closed” groups in Workplace by Facebook sites. We did indeed find that closed groups were more cohesive than open groups. However, we also found that it was less about the Open/Closed status but more about the size of the group that dictated cohesiveness. Smaller groups are more cohesive than large groups and it just so happens that on average closed groups are smaller than open groups.

Amy’s post, however, addresses knowledge sharing and getting answers to difficult questions. We might infer that the more cohesive a group is, the more amenable they might be to the above. But we can do better than that. We have recently implemented a ‘curiosity index’  in SWOOP, which simply measures the proportion of group messages that contains questions.

We have also been given access to some SWOOP customer sites, who have enabled sentiment analysis. We can therefore characterise questions according to their level of sentiment. Arguably, questions expressed with lower or more challenging sentiment might suggest more difficult questions are being posed. For privacy reasons, SWOOP does not store message content, so we have no other way to judge if a question is ‘difficult’ or not. A high response rate would therefore indicate how well these potentially difficult questions are being answered?

In summary our propositions would be:

Proposition 1: Private groups are more open to sharing than public groups.

Proposition 2: Private groups are more willing to ask the difficult questions than public groups.

Proposition 3: People are less willing to answer difficult questions in public groups than private groups.

Analytical Methodology

With quantitative research we need to operationalise the factors we are assessing with artefacts that we can actually measure. The identifies the SWOOP measures that we believe best represent the factors in our propositions:

While this is far from perfect, it is the best we can do at this time. 

Results

We selected two large organisations that have SWOOP sentiment analysis turned on. Both organisations have more than 30,000 staff. We selected a set of their most active groups, just over 300 in total. Around one third were private groups. We ran statistical T-Tests to identify those measures from the above, that were statistically different for public and private groups, with the following results:

Proposition 1: Private groups are more open to sharing than public groups

I think we could support this proposition. Even when controlling for group size, the ‘sharing’ measures favoured private over public groups. The only anomaly is the Replies/Post measure which suggests that Public groups are more responsive to posts than Private groups. Perhaps this could be attributed to the large open forums eliciting wider and more extended responses over time, than say the smaller private groups that may be more focused or single themed.

Proposition 2: Private groups are more willing to ask the difficult questions than public groups

The sentiment in questions measure could not be differentiated between public and private groups. If we believe that difficult and challenging questions have more negative sentiment (a big ‘if’), then this proposition fails. 

Proposition 3: People are less willing to answer difficult questions in public groups than private groups

When we looked at the response rate to questions asked in Private Groups vs Public Groups there was no significant difference, with Private groups on average getting 0.65 replies/question asked and Public groups getting on average 0.66 replies/questions asked. Therefore, there is no evidence to support this proposition.

Conclusions

We find that the Private groups are more open to sharing than public groups. However, despite our inclination to sense that private groups are places where more questions (both high and low sentiment) are asked and answered in private groups, we found no support in the data. We would point out that our method of relying on sentiment to classify a question as ‘challenging’ is speculative. A better approach would be to manually review, assess and flag those questions considered as ‘challenging’, to more comprehensively test Proposition 3.