February 9th, 2010 | 
Author: 
Youth social networking researcher dana boyd has observed that many people presume the way they use social networks is the way everyone uses them. “I interviewed gay men who thought Friendster was a gay dating site because all they saw were other gay men,” she says. “I interviewed teens who believed that everyone on MySpace was Christian because all of the profiles they saw contained biblical quotes. We all live in our own worlds with people who share our values and, with networked media, it’s often hard to see beyond that.”
Now picture our perspective leaving our own experiences, zooming out and up, until we can see how all the different groups are interacting on a world-wide social network. That bird’s-eye-view could be both beautiful and horrible if the resolution was clear enough. That’s what a Ramen-eating, ex-Apple engineer named Pete Warden is about to release to the public this week.
Sponsor
This Wednesday, Pete Warden will make available friend, fan page and name data about hundreds of millions of Facebook users available to the academic research community. It’s a move that Facebook has to see coming, a move that many in the data-centric community have been calling on the company itself to do for years and an event that’s complicated by Facebook’s recent privacy policy changes that have muddied the waters of right and wrong but rendered even more data left open for outside analysis.
If what people call Web 2.0 was all about creating new technologies that made it easy for every day people to publish their thoughts, social connections and activities – then the next stage of innovation online may be services like recommendations, self-and-group awareness and other features made possible by software developers building on top of the huge mass of data that Web 2.0 made public. It’s a very exciting future, and Pete Warden is about to fire one of the earliest big shots in that direction.
Nerds in Space: Social Graph Analysis For Solving Large-Group Problems
Pete Warden studied Computer Vision in college in the UK, then got into game development. After moving to LA, he spent 6 years building graphics drivers for the original Playstation and the XBox. Then he started his own independent business, where thankfully he open-sourced much of his work (something he’s still doing today).
When he found out that starting his own business wasn’t going to work with his immigration status, he was very fortunate to have also caught Apple’s eye with the software he had been releasing to the public. Apple bought his company in order to bring him on board. The proceeds of that small sale are now sustaining his next project after leaving the company.
After spending 5 years at Apple struggling to navigate the maze of people and connections and types of expertise in order to get the information he needed, Warden decided to go independent and build a company that solved exactly that kind of problem. “I can’t think of a better big company to work for, but it was still a big company,” he says. “It was hard to find the right people to talk to, whether for particular expertise or for contacts at external companies.” And so Pete Warden left Apple to build a company that would use social graph analysis to solve problems like that. He called the company Mailana.
We’ve written here a number of times about Mailana’s tool to analyze the social graph of any Twitter user. Enter the username of someone on Twitter and Mailana will show you which 20 other people they have exchanged the largest number of reciprocal public @ replies with. Find someone interesting or important? Mailana’s Twitter analyzer will tell you who they most regularly interact with. See, for example, The Inner Circles of 10 Geek Rockstars on Twitter.
Pulling Down the Facebook Social Graph
Now Warden is about to unveil a much larger project along the same vein. For the past 6 months he’s been crawling public profile pages on Facebook. He now has more than 215 million of them indexed and updated about once a month. When he began he was using webcrawling service 80legs but in time he had to build his own crawling infrastructure.
When I talked to him this afternoon, he had already begun uploading 100 GB of user data onto his server to make available for academic research starting on Wednesday. Warden says he’s removed identifying profile URLs but kept names, locations, Fan page lists and partial friends lists. All those fields of data are just waiting to be analyzed and cross referenced. That’s one very rich resource.
Yesterday Warden posted some of his own initial observations from the data on his personal blog. Those included:
- God is in almost every state in the Southern US the #1 most popular Fan page among Facebook users. Among people in the LA, San Francisco and Nevada regions? “God hardly makes an appearance on the fan pages, but sports aren’t that popular either,” Warden writes. “Michael Jackson is a particular favorite, and San Francisco puts Barack Obama in the top spot.” In the Oregon and Idaho region? Starbucks is #1.
- In the Mormon-influenced areas of Utah and Eastern Idaho, the most popular Fan pages are The Book of Mormon, Glen Beck and the vampire book Twilight, which was authored by a Mormon.
- The bulk of Warden’s posted analysis yesterday was about location networks. People in the Western US tend to have Facebook friends all over the country, people in the Southern US tend to mostly be friends with people who have remained in the same area.
Taking a Deeper Look
These observations are interesting, but they are only the beginning of what’s possible. Name, location, friends and interests are great data points to analyze. Warden has written a program that will estimate gender as well, based on names. All these data points can be cross-referenced with outside data, too. Members of Facebook’s own staff did this kind of analysis when they compared last names on the site to US census data about the likelihood of people with particular last names reporting particular racial backgrounds in order to estimate the changes in Facebook’s racial composition over time.
“I’m mostly thinking ‘What do I try first?’,” Warden says. “There’s so many interesting ways to slice the data – especially as I’m starting to get changes over time. I’m also trying to map out political networks in aggregate; how polarized the fans of particular politicians are – so how likely a Sarah Palin fan is to have any friends who are fans of Obama, and how that varies with location too. One of my favorite results is that Texans are more likely to be fans of the Dallas Cowboys than God.”
Warden says he hasn’t talked to anyone from Facebook since he started crawling the site, but he did get an email from someone on the security team asking him to take down instructions he’d posted exposing a security hole that made harvesting peoples’ email addresses easy. So the company is paying attention. “I’d love to see them put me out of business by putting decent data out there,” Warden says. He says his Amazon Web Services bill was over $5k last month.
Why is he indexing all this content and why is he going to hand it over to the academic world later this week? “I am fascinated by how we can build tools to understand our world and connect people based on all the data we’re just littering the internet with,” Warden says.
“Nobody thinks about how much valuable information they’re generating just by friending people and fanning pages. It’s like we’re constantly voting in a hundred different ways every day. And I’m starry-eyed believer that we’ll be able to change the world for the better using that neglected information. It’s like an x-ray for the whole country – we can see all sorts of hidden details of who we’re friends with, where we live, what we like.”
For a great example of the kind of social impact that data analysis can make, Warden points to some of the fascinating ways that GIS data is illuminating the intersection of race and public services. Data has shed light on social injustices for decades and measurable information about the interactions of hundreds of millions of people every day on Facebook offers opportunities to discover both good and bad news about the contemporary human condition.
Warden says he’s not yet been able to interest any investors in his ideas for businesses based on this data, so his girlfriend Liz Baumann, a former insurance actuary, stepped in to help and is now running much of the crawling. He says he’s now focused on “working on ways of presenting all this information in a form that answers questions for people willing to pay.” His first experiment along those lines is the very interesting FanPageAnalytics.com.
What does Pete Warden hope for from this week’s public release of all this Facebook data? “Hopefully I’ll get to see a bunch of interesting [academic research] papers come out of it, worst case. And I’d like to be the guy people turn to when they need stuff like this.”
Already well-respected among a fringe group of bleeding-edge geeks, we hope that Warden’s work on social graph analysis will end up impacting a far larger number of people than may ever know his name.
Source: Marshall Kirkpatrick
Posted in Uncategorized | 
If you’ve been a geek your whole life then you understand the term “Canadian girlfriend.” The Canadian (or sometimes British) love interest is the person you talk about when a member of the opposite sex inquires about your dating status. The story is that you met online, you’ve formed a solid bond and you’ll probably break up with your online girlfriend when a girl in your vicinity decides she likes you. The idea is to drive up the value of your perceived social stock. In the startup world, the same principle is used in “ham and egging.”
Sponsor
As pointed out in a recent blog post by university professor Scott Shane, “ham and egging” was first coined by Columbia’s professor Amar Bhide and Harvard Business School’s Howard Stevenson. The term refers to the technique of convincing multiple stakeholders that others are working with you despite the fact that you’re only in talks. The only problem is that most early partners only want to work with you if other reputable partners have already signed on.
Explains Bhide and Stevenson,”the ultimate ham and egging solution is for the entrepreneur to simultaneously convince each participant that everyone else is on board, or almost on board.”
However, when ReadWriteWeb spoke to MobiTV CEO Paul Scanlan about forging deals between telecom and television companies, he suggested a different tact. Although Scanlan found himself caught between partners who were skittish to sign on without the initial validation of others, he decided that rather than ham and egging, he’d build contingency clauses into contracts. Scanlan’s contracts stated that all partnerships were contingent on a set number of large-scale partners to launch. While this may not be the ideal method of closing deals, it seems like an ethical alternative to engaging in deals that begin with dishonesty.
Have you ever engaged in ham and egging and if so, was your deal a success?
Source: Dana Oshiro
Green Goose is a new financial management service that launched today, which connects sensor activity to your savings account. At first Green Goose sounded a little gimmicky. Using green Internet-connected eggs, it measures how much energy you expend on your bike or how much water you use in your shower – and transfers amounts from your checking account to your savings account based on the ’savings’ you made doing those activities.
What’s interesting though is that the savings are calculated based on the actions measured by small battery-powered, wireless sensors. You stick these sensors on your bike, thermostat, showerhead "and even your keychain."
Sponsor
Green Goose is a web-based service, along with "a very low-cost set of Savings sensors." – these are literally green eggs (see picture to the right). The web site tracks specific actions and behaviors from users – then computes that into dollars saved.
Co-Founder Brian Krejcarek told ReadWriteWeb that’s "like a Twitter feed of personal green savings."
Here’s how the sensor part works: the sensors communicate with a "Green Gateway" that then sends messages to the web site. The Green Gateway – which is also "egg-like" – has an Ethernet port that connects to your network hub via a router. The bike sensor measures miles ridden. Green Goose also plans to offer sensors for your automobile, shower (hot water), and thermostat (heating and cooling).
In the future, Green Goose might also be able to pull savings data in "from open APIs like that proposed by Google Power Meter for savings earned by using less electricity." It also plans to eventually move beyond energy to capture savings earned from making "other lifestyle decisions."
You can get started today with a "Green Goose Bike Sensor Kit," which retails for $49 plus $10 for postage. The Portland and San Francisco-based company is currently in talks with the BTA (Bicycle Transportation Alliance) in Portland and they’re already installed "in a number of coffee shops."
As well as consumers, the service is targeting employers with a "a unique sustainable savings benefit" offering for their staff. One of the features for employers is managing and auditing details for the IRS bike commute tax credit.
Green Goose is currently in pre-production and running beta trials. Right now it’s offering 100 Savings Kits for bicycle owners.
Eventually this type of connection, between sensors and mainstream services like banking, will be commonplace and probably won’t need to rely on gimmicks such as green eggs. But for now, Green Goose seems like a cute, interesting Internet of Things service for green conscious early adopters to try out.
Source: Richard MacManus
A
Source: Dave Winer
A
Source: Zee
A
Source: Alex Wilhelm
A
Source: Alex Wilhelm
Between Y Combinator’s Startup School, the influx of seed fund incubators, the list of legendary mentor / investors and the dotcom bust’s school of hard knocks, is there really any reason to go to grad school? At ReadWriteWeb we’re supportive of lifelong learning and universities that coach entrepreneurs, but a recent post by Venture Hacks founder Naval Ravikant has us wondering, “What is the value in grad school?”
Sponsor
Ravikant suggests that incubators and accelerators like YCombinator and Techstars are the new grad school.
He writes, “In some ways, it’s better” and includes the fact that unlike business schools, YCombinator pays entrepreneurs, allows founders to be their own boss and encourages original work.
In addition to Ravikant’s points, the fact that every incubator participant is connected to advisors through a financial agreement means the group may be motivated to maintain their network and share contacts.Nevertheless, before dismissing the idea of grad school altogether, it’s good to remember many of the top entrepreneurs and investors in Silicon Valley are MIT, Harvard and CalTech grads (including some of the Venture Hacks team). Perhaps the argument here is not so much about incubators over traditional institutions, but in the fact that good mentors have a stake in your success and do not rest on the laurels of a tenured position.
As a startup entrepreneur, what is the best lesson you’ve ever been taught and who taught it to you?
Photo Credit: Duncan Hull
Source: Dana Oshiro
Geolocation social networks are set to be in 2010 what microblogging was in 2008 – the next big thing. Currently the space is being dominated by Foursquare, with others like Gowalla, MyTown and Loopt trailing in its wake.
While Gowalla has secured a large amount of funding, some $8.5 million, and My Town claims more check-ins than the other services, Foursquare is happily ticking along on the seed money provided by its founders (after they sold their original effort, Dodgeball, to Google), and creating a community of developers who are eager to build secondary applications. There are two reasons Foursquare is gaining so much ground over its competitors.
Sponsor
This is a guest post by Simon Salt, the founder and CEO of IncSlingers. He is a writer and blogger whose work has appeared in a wide variety of places, including Chris Brogan’s Dad-O-Matic.com, American Marketing Association News and the Austin Realtor. He has been quoted in the Wall Street Journal on social media and is a recognized national speaker on that topic as well.
Firstly, it is available on all phone platforms. While this is also true of Loopt, the game play element of Foursquare makes it more attractive to many users. Secondly, it took the very sensible step of opening its API early on. This has generated a wave of
new secondary applications.
In addition to this, it set up a developer community forum and is extremely active there, providing support, information and assistance to developers that are creating these secondary apps. Just as with Twitter, it is not just the service itself that will create demand, but the applications that run from that service.
There are hundreds of applications that run on the Twitter API now (most of which are listed at oneforty.com). It is very likely that by the end of 2010 we will see similar numbers of applications for Foursquare, and, should Gowalla and the rest open their APIs, for those services too.
The reason geolocation social networking is so popular is quite simply its ease of use. Arrive at a destination, be that your work, the gym, a coffee shop or even the grocery store, fire up the application on your phone, “check-in”, and get points, badges and even increased status in the game for doing so.
As with Twitter, early adopters are the ones using the sites the most at the moment, but the services are rapidly spreading beyond the hard core. Unlike micro-blogging, you need no talent to be entertaining, informative or stylish – you simply check-in as you go about your day. You attract “friends” by doing this, or by connecting with people you already know, either online or in real life.
The Next Step: User- and Location-Focused
As with so many other early-stage social network platforms, extending the service with secondary applications is what makes the service truly useful.
At present, the secondary applications that are being developed fall into two main categories: user-focused and location-focused. A good example of a user-focused app is wheredoyougo. This service provides a heat map visual of all of a user’s check-ins. Another, foursqpic allows users to upload pictures as part of the Foursquare Venue Tips section, supplementing simple text tips with visuals as well. These are great apps and certainly add to the fun and extend the functionality of the service. However, the real gains are coming in the location-focused apps.
So far, the most developed of these is placewidget, which allows owners of a location to promote, via a website widget, the “Mayor” of their location on their website. Until now, any real marketing revolving around Foursquare was offline, and had to be location-specific. By bringing the ability to market both their involvement in the Foursquare community and promote a loyal customer, this widget gives a lot more power to businesses looking to leverage this type of social networking.
Foursquare recently announced it had signed two deals with media outlets. The first is an agreement with Metro, Canada’s number one free daily newspaper, to have content for venues provided by the newspaper. The second is with Bravo TV, which will include Bravo Celebrity Tips and Bravo-branded badges for over 500 locations. A game is great, but a content-rich social network is something a lot more valuable.
Photo credit: Agata Urbaniak
Discuss
Source: Guest Author
A
Source: Alex Wilhelm