For the last few years I’ve been working on mapping out social relationships in cities with my project peoplemaps.org. I had the chance to speak about this work recently at the TED Global conference, and the talk was featured this week on the ted.com website. This has led to many, many inquiries about the project, how it works, and the limits of its applicability.

So I wanted to take an opportunity to explain in high-level terms how I’m doing this work, what it tells us, and what it does not. I’ll attempt here to cover the approach in enough detail so others can reproduce it, without going into deep implementation details which are frankly not important to understanding the concepts. However, I’m happy to collaborate with anyone who would like to discuss implementations.

Theoretical Foundation

In the real world, each of us has personal relationships: our family, friends, co-workers, people we interact with in social settings, children and their parents and friends, and the like. The conventional wisdom is that we can maintain roughly 150 real-world relationships — this is called the “Dunbar Number,” named after the anthropologist who identified this phenomenon. Some people may have more, some people less, but 150 is probably average.

In an online setting, people may have many more relationships — perhaps a few thousand that they interact with in some meaningful way. However, the offline, real-world relationships that people have will likely be an overlapping subset of the relationships they have in an online setting. Online relationships are often more flexible (they can be global and operate at all times of day), though not always as meaningful. Still, if you are able to look at people’s online relationships in a specific geography, you have at least a proxy for understanding their offline, real-world relationships as well.

The ability to infer things about both offline and online interactions is derived from the principle of homophily, which sociologists identified as the powerful tendency for people to cluster into groups of people who are similar to themselves. Colloquially we know this through the saying, “birds of a feather flock together,” and it is powerfully demonstrated in network data.

So if we accept the notion that people do, in fact, have relationships that both shape and are shaped by their interactions, then it follows that there may be some ways to measure these relationships with at least some level of fidelity. Social network data appears to offer at least a window into these real-world relationships, though each dataset has biases which are not yet well understood. However, by comparing results from different networks, we can start to get a sense for what those biases might be.

This is where the state of the art for this research is right now: trying to understand how these different data sets are biased, the nature of those biases, and whether the biases are material in terms of distorting the “real world” network we are trying to understand. However, this lack of perfect understanding isn’t preventing people from using this data to do all kinds of things: from recommending movies to helping you find “people you may know” to identifying terror cells using cell-phone “metadata.” All of these activities use essentially the same approach.

These Maps Are NOT Geographic

A quick caveat: while we typically think of city maps as geographic, these maps are explicitly NOT geographic in nature. Rather, they are showing communities and their relationships to each other. The position of communities relative to the page is always arbitrary, their position relative to each other is determined by the presence or lack of relationships between them. I bring this up now to dissuade any notion you might have that these maps are geographic, despite whatever resemblance (real or imagined) they might have to geography. This all said, there are ways to tie these maps back to geography and use it as an additional investigative tool, but you should assume that all discussion of maps here is non-geographic, unless otherwise noted.

Gathering the Data — and Avoiding Bieber Holes

Depending on the dataset you are looking to explore, the exact details of how you gather data will vary somewhat; I have used data from Twitter, Facebook, LinkedIn, AngelList, email, and other sources. In all cases, you will want to gather a set of “nodes” (people, users, or companies, depending on the data source) and “edges” (relationships between them — typically “friend” or “follow” relationships.)

For the case of the city network maps, this is the approach I have used with Twitter data:

1. Define a target geography using a geographic bounding box.
2. Determine a set of keywords and location names that users inside the target geography may use to identify their location/geography.
3. Identify a set of “seed” users within the bounding box, or which otherwise appear to verifiably be within the target geography.
4. Determine which user identifiers are followed by a given user, and record that in a list that maintains the number of followers for a given user identifier.
5. When a given user identifier is followed by a number of people exceeding a threshold, request the full user information for that user to see if it appears to be within the target geography.
6. If that user is within the target geography, then feed it into step 4, requesting the user identifiers followed by that user.
7. Repeat steps 4-7 until there are few new additions to the dataset.

Doing this process once produces a “first draft” data set, which can then be visualized for inspection — to look for improper inclusions, obvious exclusions, and any particular data artifacts or pathologies.

At this stage, various problems may appear. As an example, if you are trying to visualize Birmingham UK, you will likely end up with some data for Birmingham, Alabama, due to legitimate confusion which may exist between the two communities. At this point, you can modify the test used to determine whether something is inside the target or not, and regenerate the dataset, as well as perform additional data gathering iterations to get more of the “right” data. This process typically takes a few iterations to really drill down into the data you’re looking for.

One persistent problem I’ve come across is a phenomenon called Bieber Holes, which are essentially regions of the network occupied by Justin Bieber fans. They are so virulent, and their networks so dense, that only aggressive exclusion filters can prevent the algorithm from diving down into these holes and unearthing millions of Beliebers — only a fraction of which may pass location tests. Anyway, I’ve developed good techniques to avoid Bieber holes (and similar phenomena) but it’s a reminder that when working with data from the public with algorithmic approaches, editorial discretion is required.

Laying out the Network Graph

There are dozens of algorithms for laying out network graph data, each optimized to illustrate different properties of the graph. Since I’m primarily interested in homophily and clustering, I’m looking for layouts that express communities of relationships. A good way to do that is to use a force-directed graph layout algorithm; with this approach, relationships act like springs (expressing Hooke’s law), and each user or node repels nearby nodes (expressing Coulomb’s law). By iteratively drawing the graph based on these forces, the graph will eventually reach a steady state which exhibits the following properties:

1. People with many relationships between them will be arranged into tight clusters.
2. People with the fewest relationships between them will appear at opposite edges of the graph.
3. People who have many relationships at both ends of the graph will appear in the middle.
4. Clusters with few or no relationships between them will appear very far apart on the graph.

You can think of this in very simple terms. If you have a room full of 10 people, there can be a total of 45 relationships between them (n * (n-1)/2, since someone is not friends with themselves). If every person is friends with every other person, this network will appear as a perfect and symmetric “ball” under the rules of a force directed graph layout.

Likewise, let’s suppose that same room of 10 people was grouped into two groups of 5 people, and that those two groups hated each other and refused to speak, but each member of each group knows every other member of their own group. You would see two perfect “ball” layouts (each with 10 relationships expressed), but with no connection between them.

When we visualize data from a city in this way, we are essentially measuring the separateness of communities — whether we see several of these separate groups or whether we see one unified community.

Note that in the example with 10 people in one group, we have a total of 45 relationships, while in the example with two groups of 5, there are only 20 (2 x 10). Scaling that up to a city of 500,000 people, if the city was fully meshed, there would be 124,999,750,000 total relationships, while if that city is segregated into two groups of 250,000, there can only be 31,249,875,000 relationships in each group, or 62,499,750,000 total relationships across both groups, which is a little less than half of the number of relationships possible if the two groups merged.

Detecting Communities and Adding Color

Within the network, we can use algorithms to detect distinct communities. Communities are defined by the number of shared relationships within a given subgroup. We use an algorithm called the Louvain community detection algorithm, which iteratively determines communities of interest within the larger network, and assigns a community membership to each user accordingly.

We can then assign each of these assigned communities a color. For my work so far, I have assigned these colors arbitrarily, with the only goal of visually differentiating one community from another. This helps to generate an aesthetically pleasing visual representation.

While communities often correlate to clustering exhibited by the layout algorithm, for nodes that are not clearly members of only one cluster, color can be used to indicate their primary community affiliation. This is helpful in the visualization because generates blended color fields that can give a sense of the boundary between two communities.

For example, a group primarily concerned with politics (blue) and a group primarily concerned by music (yellow) may have a mix of both blue and yellow nodes in the space between those distinct communities, and you can get a sense that those people are very interested in both communities and which their primary affiliation might be.

Assigning Node Size with In-Degree

In a graph of Twitter users, it can be helpful to indicate how many people are following a given user within our specific graph (note that this is calculated for our subgraph, not taken from a user’s “global” follower count as displayed by Twitter.)

We can do that by making the “dot” associated with each user bigger or smaller based on the number of followers. This has no real effect on the shape of the network, but it can be helpful in determining what kinds of users are where, and how people have organized themselves into communities of interest.

Determining Community Interests

After you have a colored graph with communities and clusters, it’s time to try to figure out what these clusters seem to be around. The first thing to do is to start by manually inspecting nodes to see who they are — often starting with the biggest nodes first. Typically you’ll find that people often organize themselves into groups like these: sports, music, mainstream media, politics, food, technology, arts, books, culture, and the like. These clusterings vary somewhat from city to city, but you’ll see some common patterns between cities.

After communities are detected, we can start to monitor traffic coming out of these communities to look for topics of conversation and other characteristics which may be helpful in explicating the observed clustering. For instance, we can gather a corpus of Twitter traffic for each community, recording:

1. hashtags
2. commonly shared links
3. languages in use
4. operating systems in use (desktop vs. mobile et al)
5. client software in use
6. geographic coordinates (as provided in GPS)
7. age of user accounts
8. user mentions

With this kind of data recorded as a histogram for each community, we can start to get a pretty good sense that a given group is mostly concerned with sports, music, politics, and the like. By working with a collaborator who is well-versed with the culture of a given place, we can also get a sense of local subtleties that might not be immediately obvious to an outside observer. These insights can be used to generate legends for a final graph product, and other editorial content.

Investigating Race

Another phenomenon is very apparent in American cities: people separate by race. American cities like Baltimore and St. Louis are polarized into black and white communities, with some people bridging in the middle. These cities present roughly as eccentric polygons, with differences in mesh density at each end of a racially polarized spectrum. Relatively prosperous European cities, like Munich or Barcelona, present more like “balls,” with no clear majority/minority tension displayed. Istanbul, by contrast, shows a strong divide between the rich, establishment and a large emergent cohort of frustrated young men.

In many American cities, we do observe strong racial homophily in the data. For example, in the data for a city like Baltimore, people at opposite ends of the spectrum are generally strongly identifiable as “very white” or “very black.” This is a touchy subject, and it’s difficult to discuss this topic without offending people; however, what we are aiming to do here is to try to understand what the data is showing us, not make generalizations or prescriptive measures about race.

One way to ground this discussion about race is to look at the profile photos associated with user accounts. We can gather these photos in bulk, and when displaying them together, it becomes quickly clear that people have often organized themselves around skin color. When they have not organized themselves around skin color, they are organized around other cultural signifiers like fashion or style. This can give us a concrete sense that certain communities consist primarily of one race or another. However, this is not to suggest that outliers do not exist, or to make any statement either about any given individual — and certainly does not suggest an inverse relationship between race and the ability or proclivity to participate in any given cluster or clusters.

The Final Product

Once we have refined the data, identified communities, investigated trends and topics within communities, and potentially looked at photos, profiles, and other cultural signifiers, we are in a position to annotate the network map with editorial legends. This is a fundamentally human process. I generally use a tool like Photoshop or Keynote to annotate an image, but this could be done in a number of ways. Once this step is done, a final product can be exported.

This entire process of taking data from one or more biased sources, refining data, and then using a human editor who also has a bias produces a final end product which is comparable to an “op-ed” in a newspaper: it’s an expression of one possible mental model of the world, informed by a combination of facts, errors, and pre-existing opinions. It’s up to the reader to determine its utility, but to the extent it offers a novel view and the reader deems the data and editorial biases acceptable, such renderings can be an informative lens through which to see a city.

A Note About Bias, Inclusions, and Exclusions

A common criticism of large-scale social network analysis is that it is not representative because it is biased in some way. A typical statement is, “I don’t use Twitter, so the results don’t include me, therefore I question the validity of the approach.” Likewise, people may say the same thing about LinkedIn or Facebook, or start into a long explanation of how their online habits are very different from their offline reality. These are important facts to consider, but the question is really whether network analysis is representative enough to start to deliver information that we didn’t have until now.

To answer this, it’s helpful to think about this in terms of recent history. In 2004 or 2007, it probably wasn’t helpful to say anything in particular about data from social networks, because none of them had enough penetration to deliver insights beyond a very biased community of early adopters: whether it was geeks in the case of Twitter, or young music fans in the case of MySpace, or college students in the case of Facebook.

As these networks have continued to evolve, however, their penetration is increasing rapidly. They are also accreting a great deal of historical data about people and their position in social networks which give us clues as to their offline interactions. This accretion of data will only continue and begin to paint a more complete and multi-dimensional picture of our culture — especially if you correlate the data from multiple social networks.

I believe we are now at a point where these data sets are large and detailed enough to offer important insights about our “real world” culture. This belief is based on two facts. First, it’s possible to get a good working image of a community by gathering data over just a few hours. While we make every effort to gather as much data as is realistically possible before making statements about a community, the effect of adding more data is additive: we add members to the community, but we do not fundamentally alter its shape.

To understand this effect, it is helpful to consider an analogy from astronomy. A more powerful telescope can yield a better, sharper image of a star formation, but it neither changes the shape of the star formation or our basic understanding of its structure. Better tools simply yield more detailed data. I believe we are at a point where we have enough data to begin to understand structures, and that more data will yield more detailed understanding — but not alter the fundamental shapes we are beginning to uncover. This line of thinking is helpful in comprehending “exclusions.”

On the subject of “false inclusions,” these are generally sussed out in the layout and review process, but all of the graphs I have produced have generated a limited number of false inclusions as a practically inevitable artifact of the process. As a result, any statements one may make about a specific individual based on this kind of analysis may or may not be valid: they should be viewed through the lens of the biases disclosed alongside the visualization. However, repeated experience has shown that removing false small number of false inclusions does not have a material effect on the community structure.

On the subject of “network bias” (I don’t use x, therefore conclusions are not valid), when comparing data from Facebook, Twitter, LinkedIn, AngelList and other sources, the same patterns of homophily are exhibited. While each network has its own bias (LinkedIn towards professionals, Twitter towards both youths and professionals, Facebook towards Grandparents), if we limit analysis to a given geography, we will inevitably see homophilic tendencies which are correlated in each network.

While it’s difficult to speak about this in detail yet, as obtaining full comparable data sets from each source is currently quite challenging, early investigations indicate that the same patterns are exhibited across all networks. This squares up with the notion that what we are really doing in examining these networks is but a proxy for real world relationships. Ultimately, these networks must converge into something that approximates this abstract, Euclidian reality, and as we get more data and correlate it, it’s likely that a unified data set will closely reflect the actual geometry of our communities.

Future Research Directions

This kind of analysis is forming the basis for the emergent field of “computational sociology,” which is currently being explored by a variety of researchers around the world. This work has a number of important implications, and poses questions like:

• What do we mean by diversity? If we should be looking to bridge networks, is race a helpful proxy for that function? Or should we be looking to develop new better measures of diversity?
• What is the nature of segregation? Is physical segregation a product of our social networks? Or is it a manifestation of them?
• What is the role of urban planning? Is our social fabric something that’s shaped by urban planning, or are our cities simply a manifestation of our social fabric?
• What kind of interventions might we undertake to improve a city’s health? Should they be based first on creating and improving relationships?
• What is creative capital and how can we maximize it? If creative capital is a byproduct of relationships between people with diverse backgrounds, then we should be able to increase creative capital by orders of magnitude by bridging networks together and increasing the number of relationships overall.

The level of detail we are able to extract from network data in cities is more detailed than any data source we have ever had, and can supplement many other indicators currently used to measure community health and make decisions about resource allocation. For example, the census may tell us someone’s race, family name and physical address, but it tells us very little about their participation in the social fabric. And if we believe that the city is primarily manifested as the sum total of social relationships, then clearly data surrounding social interactions is more useful than other attributes we may harvest.

Discerning Stable Structures vs. Topical Discussions

Likewise, many researchers have been using social network data like Tweets to characterize conversations around a topic; and it is true that by harvesting massive quantities of Tweets, one can discern trends, conversation leaders, and other insights. However, this type of analysis tends to be topical and shift a great deal depending on whether people are active online, and contains biases about why they might be active online. Regardless, this kind of research is well worth understanding, and may ultimately lead to a better understanding of network structure formation. However, I want to differentiate it from the kind of inquiry I am pursuing.

So, rather than “you are what you tweet,” I believe the truth is something more like “you tweet what you are.” Network follow structures tend to be very stable; while they do change and evolve over time, one interesting feature of the process I am using is that it tends to be very stable over time and the results of analysis are repeatable. That is to say, if I analyze Baltimore one day, and then repeat the analysis a month later, I will obtain a comparable result. This is not likely to be the case with semantic analysis as topics and chatter may change over the course of a few weeks.

Tracking (and Animating) Changes Over Time

Because network structure analysis is fundamentally labor intensive and somewhat difficult to compute, it’s difficult to automate and to perform on an ongoing basis. For example, it would be nice in Baltimore to know whether things are getting “better” or “worse.” Right now, only by taking periodic snapshots and comparing them can we get a sense for this. It would be helpful to be able to animate changes over time, and while this is theoretically possible, tools to do this have to be built by hand right now. My hope is to apply such tools to the process and dramatically increase our ability to monitor network structure in real time, and spot trends.

Ultimately, this will give us some ability to see our social structure change in near real-time, developing a sense as to whether interventions are having a positive effect, or any effect at all.

Healthy vs. Unhealthy Network Patterns – and Brain Development

Dr. Sandy Pentland, an MIT researcher who is perhaps the leading figure in the field of computational sociology, has suggested that there are certain patterns that characterize “healthy” networks, as well as patterns that characterize “unhealthy” networks.

Healthy networks are characterized by:

• Frequent, short interactions
• Broad participation by all nodes and meshing
• Acknowledgement of contributions
• A tendency to explore other parts of the network

By contrast, unhealthy networks exhibit the opposite patterns: broadcast vs. peer-to-peer interactions; fewer network connections; isolated subnetworks; a lack of exploration of other parts of the network.

Perhaps the most telling finding is that these patterns are not limited to just human networks, but also appear in other colony-based organisms, like bees. It appears that there are universal patterns of network health that apply to life in general.

The other important finding is that lack of network exploration affects brain development in young people. Specifically, young people who grow up in an environment where network exploration is not valued tend to exhibit structural changes in the brain which appear to dampen their desire to explore networks as adults. This produces a multi-generational effect, where children who grow up in isolated networks tend to persist in isolated networks, and to pass that on to their children.

This suggests that one possible intervention is to promote network exploration at an early age across the entire population. How we might do this is certainly open to discussion, but it seems to be a very powerful tool in breaking down divisions in our social networks.

What’s next?

As this project advances, we are gathering ever-larger datasets. This requires more and more computational power and distributed algorithms for visualization. If this is something you’re also working on, I would like to talk to you — please contact me. There are some interesting challenges in scaling this up but there are interesting opportunities emerging to apply these approaches!

Mapping Your City

We have a long list of places that we’re looking at mapping, and trying to prioritize opportunities. If you have a mapping project you would like us to consider, please contact us. We will try to get back to you as quickly as possible, but in general, we’re looking for projects that can make a serious social impact. It takes a serious amount of effort to generate these maps, so be thinking about possible partners who could potentially fund this work and help advance this science.

The 2011 Mayoral contests represent a unique opportunity to make American cities work again. Cities have already begun an inexorable return to relevance as refuges from crushing commutes, and as havens of culture and innovation. Our economy is increasingly hitched to our ability to develop and capitalize on innovative ideas, and that innovation can’t happen when folks are trapped in their cars and isolated in the matrix of suburban sprawl. Cities are the American future.

But in the early 1970’s, they were left for dead: victims of race and class warfare, they became abandoned places – a place where people work or would go to the symphony, but not places to build a life or raise children. Formerly walkable, livable cities degraded into a-la-carte destinations you could get into and out of quickly as 1950’s visions of suburbia gained dominance.

With this shift, cities’ political influence waned, and city politics evolved into a top-down enterprise. Power brokers, political clubs, and church groups conferred power on those who would play the game and wait their turn. In Baltimore, city politics became either a launching pad for state office, or a refuge of scoundrels whose city fiefdoms became ends in and of themselves. Instead of working for Baltimore, all too often our politicians have tried to enrich themselves at its expense. With minimal popular interest and the atrophy of the press, there has been increasingly less oversight. So the machine has lumbered on – unencumbered by the tempering force of investigation, new blood, or real political imagination.

In other contexts, leaders are judged on their ability to lead and deliver tangible improvements. But in our cities, it has become enough for our politicians to just not screw things up even worse than they found them. Enough isverigeapotek.com. It’s time to move forward again.

In 2010 we saw some new trends: long-term incumbents who fit the old standard – of merely not being demonstrably corrupt or incompetent – were booted out. And not because of typical anti-incumbent anger, but because people saw something else: that maybe we could demand better.

In Baltimore, 27 year-old newcomer Bill Ferguson delivered a decisive blow to 27-year incumbent State Senator George Della. Gregg Bernstein defeated long-time incumbent Baltimore City States Attorney Patricia Jessamy. These races shared two things in common: no one thought they could upset the machine, and they used the Internet to organize financial and ideological support.

The simultaneous rise in the demand for urban living along with the use of the Internet for political and community organizing will usher in an era of unprecedented change in American cities. With the 2010 races, the old system was put on notice; in 2011 it will begin to be dismantled.

I support Otis Rolley in his candidacy for Mayor of Baltimore in 2011. At 36, Otis is part of the new guard. He’s qualified – he has a masters’ degree in City Planning from MIT. He has been in Baltimore since 1998. He served 10 years in the public sector and two in the private sector. As an executive, he led the Baltimore City Department of Planning and – shockingly – produced the city’s first actual master plan in 39 years.

In his time at Planning and as a Chief of Staff, Otis was struck with one question: can’t we do better than this?

Indeed we can. Leadership is about creating a culture based on shared values. We need a leader who is willing to stand up for his values and the values of people who care and work hard, and not allow entrenched career “slugs” to dilute those efforts. He proved he could do this at the Department of Planning, empowering those who had a vision for the city, pushing out those that did not.

But while Otis was able to turn around a non-performing department and produce a workable plan for the city, he ultimately realized that the only way to see its recommendations executed was as Mayor. We should give him this opportunity.

Otis can turn around our city the same way he turned around a department: by creating a new culture. Frankly, there are a lot of people in city government who should be looking for other kinds of work. We can start there.

Otis understands that we need to start allocating our resources differently. Economic development has for too long been about big projects, like the currently proposed $900 Million Baltimore Arena redevelopment. While this plan would assuredly enrich some developers and provide ample future backing for political operators looking to entrench themselves for a lifetime in Maryland politics, we should instead be thinking about new ways to capitalize on Baltimore’s biggest economic development assets: its people and its fortunate geography.

If instead we were to invest $900 Million in the infrastructure to support entrepreneurial enterprises and startups, we could potentially create tens of thousands of jobs across a wide range of income levels. A new startup-friendly Baltimore could outperform other regions in terms of standard and cost of living as well as access to a world-class workforce. A strategic focus on manufacturing, both large and small using the latest technologies, could restore what was once a thriving middle class. Arenas, convention centers, stadiums and hotel subsidies just deliver more jobs that don’t even pay a living wage. Otis knows we can do better.

In 2011, we have a choice: do we want to be a good city, or a great city? Otis has a vision that he will articulate over the coming months as part of what should be an open and healthy debate around the future of our city, and not about personal politics. As I have come to know Otis over the past 14 months, I am confident that he is the right leader for Baltimore’s future. If you give him an opportunity to serve, you will not be disappointed.

Baltimore is Otis’ first priority. He has no aspirations for higher office. He wants to work for Baltimore and for all of you. In 2011, we have the wind at our backs – cities are on the upswing, and the Internet is connecting us in unprecedented ways. It’s time to take back our cities and make them the vital, beautiful, functional, and inclusive places we all know they can be. Otis Rolley can help us do that. This is Baltimore’s moment; let’s seize it together.

You can support Otis Rolley in 2011 by visiting his campaign website (http://otisrolley.com) and by attending the January 11th performance by Bill Cosby in support of his candidacy. Follow Otis on Twitter at @otisrolley and on Facebook at http://facebook.com/otisformayor.

Also check out Otis’ talk at TEDxMidAtlantic on November 5, 2010:

When I was about four years old, my parents asked me what I wanted to be when I grew up. My response? A cashier. Why? They were the ones who got handed all the money.

Today, when people cite crime and education as the two major problems facing America’s cities, the knee-jerk response is to “protect city education budgets” and “put more cops on the street.” This is the same kind of simplistic logic I used as a child, and it’s just as wrong.

If this was how things worked, the safest cities in the world would be populated entirely by police, and the highest levels of education would be attained by those countries who spent the most on teachers and schools. This also is not true. Excluding repressive regimes, the areas with the least crime and most educated populations tend to be places where all citizens have access to the same opportunities.

Access to Opportunity

The new film by Davis Guggenheim, “Waiting for Superman,” chronicles a year or so in the life of a handful of students of different backgrounds as they struggle to get access to the educational resources they need to thrive. The heart-wrenching conclusion shows these kids – all but one – get denied that opportunity by a system that is clearly broken and unfair. 720 applicants, 15 spots. You get the picture.

If America is going to have a public school system, this kind of unfairness should not be tolerated. What happens to the kids that don’t get into the schools that can help them thrive? Should people have equal opportunity? Most people would say “yes.”

How We Got Here

Economic vitality is a kind of spotlight: it shines a light on things that need fixing, and provides the funds and political power to do so. Since our cities were torn apart by race riots and the global consolidation of manufacturing, the resulting precipitous decline in economic health has meant that cities have operated substantially in the dark. Watchdogs have been absent, and grassroots efforts have been underpowered.

American cities have reached a kind of feudal equilibrium. Politicians in power have little incentive to promote the kind of broad-based economic growth that could ultimately result in their ouster, but they can’t let things deteriorate so badly that everyone leaves – also stripping them of their power. And so American cities walk the line: with crime, schools, drug use and taxes locked at levels that are tolerable to just enough people that they are still worth milking, all while politicians hand out favors to power-brokers and childhood friends. Enough.

Ending the Abuse, from the Bottom Up

I wrote in my previous post that cities are now prime locations for idea-based industries. Over time this will mean an influx of wealth into cities as well as an increase in poorer populations in the suburbs.

Economic vitality in our cities borne from idea-based industries will result in a demand for accountable leadership and provide new levels of participation. In short, the feudalism will end when creative people begin to use their economic power to demand real change. The 40 year free ride is over.

Too many people in cities have resigned themselves to the idea that politics is a top-down enterprise — that it’s primarily influenced by the machine, by power-brokers, by community leaders, or by churches. Or that there’s a “turn based system,” where everyone who serves is given an equal shot unless they do something wrong.

That’s just wrong. American city politics from now on will be a bottom-up grassroots affair dictated not by the economics of writing checks to campaigns, but by the interdependent economics of jobs and a shared vision for the future of places that people care about.

Restoring Trust

To be workable, all power relationships must be a compact founded on shared values. Kids trust teachers that have their best interests at heart. Citizens trust cops who behave consistently and fairly. People trust politicians who put the civic interest ahead of their own.

There is nothing that ails us that cannot be fixed by restoring these trust relationships.

In “Superman,” Guggenheim asserts that bad teachers are kept around out of a “desire to maintain harmony amongst adults.” It’s difficult to stand up and fight to end someone’s teaching career, but it’s what’s required. To fail to do so is immoral.

It’s easier to keep getting a paycheck than to make enemies. And certainly there are dozens of systemic problems that make firing teachers very difficult. But that’s all that’s wrong with our schools, our police and our politicians: a simple failure to defend our core values.

And if we cannot agree on those core values – if a desire for personal gain exceeds a willingness or ability to serve the public – then those people deserve to be called out.

The Game Is Up

To be an old-school politician in a major American city today is to be in the way of a major cultural shift. Idealistic, intelligent, educated millennials armed with 21st century political weapons are coming, and they are going to ask why?

Why is the city so screwed up? Why are these jokers in power? Why are these incompetent teachers shuttled between schools? Why is more money spent on development deals for cronies than on parks? And why the hell can’t we clean up the blight that makes any trust or pride impossible?

Should we spend more money on schools or police? More than likely, all we need to do is let smart young people start asking questions. Crime and education take care of themselves if those that have violated the public trust can be removed from power. And with a little attention and common sense, we can ensure that more kids have a shot at the same opportunities.

Because in the end, crime, education, and blight are really just one problem, and it can be cured at least as quickly as it developed.

Some have predicted that high energy costs, either due to decreasing supply of oil or costs associated with carbon emission mitigation, will soon push people out of their cars and onto public transportation.

But there’s something else happening: people are getting sick of spending time in transit at all, making city living increasingly attractive. We are now increasingly able to infill “scrap-time” during our days with useful activity, and location-based social networks make it possible to maximize personal connections as we move about. We are engineering our own serendipity to generate real value from every moment. Why would we squander that potential by spending time in transit of any kind?

In the future, I predict:

• Air travel will be reserved for trips greater than 500 miles
• Trains will be used for trips less than 500 miles
• Bicycles, walking, and local transit will be used within cities
• Any local trip longer than 20 minutes will be seen as burdensome
• Cars will be seen as a luxury to be used for road-trips + utility hauling

And again, this will not happen due to fuel scarcity alone – it will happen because people demand it; and I’m not talking about you – but your kids and grandkids.

Regardless of what happens with fuel prices, we know that roads do fill to their available capacity. And then that’s it. Expansion does not help for long, because roads then fill to whatever capacity is available and development occurs until roads are too broken to use.

Roads are also increasingly expensive to build. A major construction project such as the popular-but-doomed Intercounty Connector in Maryland will cost over $2.6Bn to build. Is this a good long-term use of resources? Seems to me we’re throwing a bone to some 55 year-old commuters who have been annoyed with the state of the Washington Beltway since it was built, and this is the only solution they thought could fix it. Enough with the reductionist, idiotic causal thinking already: it’s a dumb idea. I don’t begrudge it, but in the long term, who cares?

The idea-driven creative industries that America has hung its hat on can only thrive in cities, where people can get together and trade ideas freely. Any barrier to that exchange lowers the potential net economic value. Put simply, all of this kind of creative work will happen in cities. Period. Because if we don’t do it in cities, we won’t be able to compete with our peers around the world who will be doing this work in cities.

So, what of the suburbs? In many European cities, the urban centers have been long reserved for the upper-class elites; poorer immigrants, often Turks and other Islamic communities, tend to inhabit the outer rings of the city – denying them crucial access to economic opportunity. This kind of social injustice is baked into many European cultures; in France, you are simply French or not French, and no amount of economic mobility will allow someone who is not of that world to sublimate into it.

This is not the case in America. We are all Americans, and even marginalized citizens are able to fully participate in all levels of our culture – though certainly there is social injustice that must be overcome.

Over the next 50-75 years, there will be a net gain of wealthier people in America’s cities and also a net gain of poorer people in our suburbs. This will be a natural byproduct of an increasing demand to be in cities, and an increasing (and aging) suburban housing stock coupled with roads that no longer function.

To fulfill our challenge as Americans, we must use these dual gradients in our cities – the inflow of the rich and the outflow of the poor – as an opportunity to maximize social justice. By avoiding flash-gentrification and fixing education as we go, we can in a span of 20-40 years (1-2 generations) offer millions of people a pathway into new opportunities that stem from real, sustainable economic growth; all the while realizing this is going to mean more color blindness all around – and that suburbs will generally be poorer than the cities.

In my home state of Maryland, the only foreseeable damper on this force is the federal government and the massive amount of money it injects into industries like cybersecurity and other behemoth agencies like the Social Security Agency.

Because these agencies and the companies that service them generally do not have to compete globally to survive, they can locate in the suburbs and employ people that live in the suburbs – and subsidize all of the inefficiency, waste and boredom that comes with that.

This is nothing but a giant make-work program and its benefactors are little more than sucklings on the federal government’s teat, which is spending money that will likely come straight out of your grandchildren’s standard of living. Right on.

Cybersecurity, for all its usefulness in possibly maybe not getting us blown up by wackos (bored wackos from the European Islamic suburbs, I’ll point out), is  nothing more than a tax on bad protocol design. For the most part it doesn’t create any new value. In the end, we’ve got suburban overpaid internet engineers fighting an imaginary, boundless war with disenfranchised suburban Islamic radicals. Who’s crazier?

Lastly, for all of you who think I’m wrong or resist these predictions because you personally “wouldn’t do that” or can produce one counterexample, I ask you: Are you over 35? If so, your visceral opinion may not matter much. I fail this test myself, but I believe my argument is logically sound and is based in the emerging attitudes of young people.

The future will be made by people younger than we, and based on everything I can see, we are on the cusp of a major realignment of attitudes and economics in America.

It won’t be too much longer til active, entrepreneurial creative professionals (black and white) in our cities look at the suburbs (black and white) and decry the entitlement culture of the suburban welfare state.