How Tweets Give Super Accurate Morning Traffic Predictions

"It is super exciting to see this method leading to better predictions of morning commute traffic as late as 5 AM, and I believe this can swiftly be deployed in many of our transportation management centers," says Sean Qian. (Credit: Shutter Runner/Flickr)

Researchers have used information extracted from tweets to provide unparalleled accuracy for predicting morning traffic patterns.

The morning commute period is one of the busiest times of day for traffic; however, it has also proven to be the most difficult time to predict traffic patterns. This is because most methods for traffic prediction rely on having a consistent flow of traffic data from the time leading up to the predicted period.

The majority of people, however, spend the time preceding their commute sleeping or performing their morning routines at home, leaving a large gap in predictive traffic data.

The researchers’ method solves this problem by pulling data from tweets sent between the evening prior and early morning of the following day. They first used Twitter’s application programming interface (API) to identify tweets within a given area (in this case, the city of Pittsburgh) with geotags indicating from where they were sent. They then used another application called Twint, a web scraper, which pulled other posts from users with geotagged tweets, to create a better picture of the times and general area within which that user was active. All data was anonymized and stripped of any personally identifiable information prior to publishing.


innerself subscribe graphic


“We argue that tweets capture three types of useful information for explaining next-day morning traffic, which includes people’s sleep-wake status, local events, and (planned) traffic incidents,” authors Sean Qian, associate professor of civil and environmental engineering, and Weiran Yao, Qian’s PhD student, write.

Further augmentation of this dataset allowed the researchers to extract additional information. Using language analysis, the team identified search terms that might indicate a traffic incident. This includes not only accidents, but also planned closures or large events like a concert, sports game, or holiday celebration.

Simple personal tweets like “Had a blast at the Pirates game!” or “This fender bender ahead is going to make me late,” can actually provide crucial information, especially when tagged with a geotag or informed by other tweets from that user. Further data was also pulled from official accounts, such as news outlets and local government, which often tweet direct reports on accidents and planned closures.

When combined, these methods provide a large dataset of information that indicated the geographic distribution and sleep/wake time of likely commuters, as well as both planned and accidental traffic incidents that may affect their commute. This bridged the information gap created by the overnight lull in traffic.

With this information, Qian and Yao were able to provide traffic predictions for Pittsburgh’s morning commute period with previously unseen accuracy and have created a comprehensive framework for predicting morning traffic conditions in urban areas.

This information also allows them to start making observations and predictions on a larger, day-to-day scale. That includes finding that Pittsburgh’s morning traffic was generally more congested on Tuesdays, Wednesdays, and Thursdays, which could enable transportation agencies to better manage the morning commute. These kinds of observations—previously impossible, due to the inability to accurately predict morning conditions—may inform larger decisions in travel demand management, signal timing control and personal destination routing.

“This research leverages machine learning and big data to understand human behavior while preserving individual privacy,” says Qian.

“It is super exciting to see this method leading to better predictions of morning commute traffic as late as 5 AM, and I believe this can swiftly be deployed in many of our transportation management centers.”

About the Authors

Their results appear in Transportation Research. - Original Study

break

Related Books:

Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones

by James Clear

Atomic Habits provides practical advice for developing good habits and breaking bad ones, based on scientific research on behavior change.

Click for more info or to order

The Four Tendencies: The Indispensable Personality Profiles That Reveal How to Make Your Life Better (and Other People's Lives Better, Too)

by Gretchen Rubin

The Four Tendencies identifies four personality types and explains how understanding your own tendencies can help you improve your relationships, work habits, and overall happiness.

Click for more info or to order

Think Again: The Power of Knowing What You Don't Know

by Adam Grant

Think Again explores how people can change their minds and attitudes, and offers strategies for improving critical thinking and decision making.

Click for more info or to order

The Body Keeps the Score: Brain, Mind, and Body in the Healing of Trauma

by Bessel van der Kolk

The Body Keeps the Score discusses the connection between trauma and physical health, and offers insights into how trauma can be treated and healed.

Click for more info or to order

The Psychology of Money: Timeless lessons on wealth, greed, and happiness

by Morgan Housel

The Psychology of Money examines the ways in which our attitudes and behaviors around money can shape our financial success and overall well-being.

Click for more info or to order