Interesting Tasks

• Explore the spatial features of stations. Find out how the location affects the check-in and check-out flow of the station.

• Identify temporal patterns of rides. Visualize the sharing bike dynamics of station states and trip circulation patterns. Study the time effect and user type influence on the temporal travel patterns.

• Represent flows between origin and destinations. Study features of the most popular route.

• Understand how station roles into origin and destination changes through the day. Cluster all the stations into several groups according to their flow dynamics.

Description of Dataset

The data used in this project is provided by the Toronto Parking Authority and publicbikesystem and contains information about complete sharing bike rides in 2016 Q4. The data comes in .csv file with a total size of 29.7 MB and contains over 140 thousand trips. Each ride record consists of: Trip_id, trip_start_time, trip_stop_time, trip_duration_seconds, from_station_name, from_station_name_id, from_station_name_lat, from_station_name_lon, to_station_name, to_station_name_lat, to_station_name_lon, user_type, distance. Riders are anonymized to protect their privacy.

The following table illustates a small slice of the dataset.

Station location and degree

First, all stations and associated wards are ploted on the map. Leaflet is used in this project. It’s an open source JavaScript library for interactive maps. The shapefile of Toronto Ward is found from Open Data - City of Toronto. Here, 25-Ward Model is used instead of 47- Ward model, because it is more up-to-date.

All the wards are colored by the total population of it. Ward population data are found from Individual Ward Maps 25 Ward Model.

From the map, it is important to note that the sharing bike stations spread out from the city center to the surrounding wards. Spadina-Fort York, Toronto Center, and University and Rosedale enjoy the major station resources.

The dots areas are proportional to the total degree of each stations. Here, degree is the concept from network analysis. Because the sharing bike system is a directed graph, there are in-degree, out-degree, and total degree for each stations. By definition, degree indicates the number of lines that enter or go out from the nodes. In the sharing bike case, degree illustates the relationship between that station to the others. The station with a higher degree is associated with more stations. However, in my project, the location density of the stations affects the degree of the stations. Stations located in the ward concentrated with more stations are more likely to have a higher degree.

The color of the dots is determined by whether the total number of check-in bikes covers the one of check-out. Assume that there are unlimited docks in a station, so there will not be any space problem about how to check-in a bike. I focus on the limitation of the number of bikes offered in station. The red dots are the stations where the check-out number goes over the check-in number. These stations tend to locate in Spadina-Fort York. Special attention should be paid to these stations to make sure riders can find a bike as needed.

## OGR data source with driver: ESRI Shapefile 
## Source: "/Users/apple/Documents/uwaterloo/stat842/final proj/WARD_WGS84.shp", layer: "WARD_WGS84"
## with 25 features
## It has 9 fields
## Integer64 fields read as strings:  AREA_ID

What stands out from the map above is that ward population is not significantly related to the number of bike stations in that ward. The number of stations in each ward varies a wide range. However, the population in every ward is similar to each other. University and Rosedale has 45 bike stations, but its population is not large. Meanwhile, Beaches and East York has relatively large population but few stations. We also count the bike sharing number in each ward.

Taking into consideration the huge difference in number of stations in each ward, it’s not surprising to find that the ride spread very unevenly in each ward. Spadina-Fort York, Toronto Center, and University-Rosedale enjoy a major share of the total rides. However, the ride records per station (ride rate) is low in Beaches-East York. It’s beyond my expectation that though there are only four stations in Parkdale-High Park, the ride rate is pretty much higher than Beaches-East York.

Descriptive Stats

First, the overall trend of all stations all over the fourth quarter in 2016 is studied.

From Oct. 1st, 2016 to Dec. 31st, 2016, there were 142,866 sharing bike trips in Toronto. The average trip duration is 790 seconds (about 13 mins), and the average trip distance is 1656 meters, which implies that customers tend to riding sharing bike for short trip.

Overall trend

I also compare the travel patterns all over the three months between total riders, casual riders, and member riders. There is a decreasing trend in all the three lines. After December, the usage of casual riders dropped down to near zero. Meanwhile, the usage of member riders experienced a sharp decrease, and the size of member usage in December shrank to one third of the one in October. It might be related to the harsh winter climate in Toronto. Member usage significantly dominates the everyday bike flow. The member line is over two times higher than the casual line. Usage of members shows some periodic pattern, while usage of casual customers is more stochastic.

Flow threshold selection

The threshold of flow \(d\) influences the flow structure and computation intensity. If \(d\) is too small, I will have difficulty in recognizing the structure in the trip flow. If \(d\) is too large, it requires more computational power. I investigate the quantiles of the bike flow \(n\) and find that it has a right-tailed density. I select the \(20\) to be the threshold. \(20\) is near the \(90\%\) percentile. I abandon the irrelevant trips and concentrate to analyze the top \(10\%\) welcomed routes.

The flow map shows the Euclidean travel routes between two stations. Each line stands for one specific route. The thickness of the route is proportional to the total trip number in each route.

What are the top 10 most popular routes?

The top ten most popular route are displayed below.

It’s interesting to find that the most popular route is a loop, starting from and ending at Bay St / Queens Quay W (Ferry Terminal). Union Station is also a popular station since it appears in \(6\) routes. The tenth most popular route is another loop route, starting from and ending at York St / Queens Quay W.

I check the details with the loop at Bay St / Queens Quay W (Ferry Terminal). The trip duration is even ten times longer than the average trip duration in this dataset. On average, riders will spend over 2 hours in this loop route, which implies that they might go around a big circle before they go back and return the bikes at the start point. This data seems to suggest that tourists tend to rent a sharing bike as a Casual rider to enjoy the beautiful view around the city.

I also plot the line chart of the average usage of casual riders and member riders during a day. I define the usage as the number of check-out bikes. The casual riders make up the majority of the customers in this loop route. The casual usage starts to increase from the noon 12:00, and remains at a high level from afternoon 16:00 till evening 21:00.

Cosine distance

Hierarchical clustering methods are used to find the station with similar temporal demands. When clustering, I use cosine dissimilarity to calculate the distance between each station.

\[ cos(Station_{k_i},Station_{k_j}) = \frac{\sum_{t=1}^{24} Rent_{tk_i} \times Rent_{tk_j}+\sum_{t=1}^{24} Return_{tk_i} \times Return_{tk_j}}{\sqrt{\sum_{t=1}^{24}(Rent_{tk_i}^2+Return_{tk_i}^2)}\sqrt{\sum_{t=1}^{24}(Rent_{tk_j}^2+Return_{tk_j}^2)}} \]

\[ d_{cos}(Station_{k_i},Station_{k_j})=1-cos(Station_{k_i},Station_{k_j}) \]

where \(cos(Station_{k_i},Station_{k_j})\) suggests the cosine similarity between \(station_{k_i}\) and \(station_{k_j}\), \(d_{cos}(Station_{k_i},Station_{k_j})\) suggests the cosine distance between \(station_{k_i}\) and \(station_{k_j}\).

I get the inspiration of using \(cos(Station_{k_i},Station_{k_j})\) from text clustering. In text clustering, each element \(X\) contains the count of some word for that document. In the sharing bike case, each element \(Station_{k_i}\) contains the count of check-out and check-in bike during certain time window. If \(cos(Station_{k_i},Station_{k_j})\) is large, the temporal pattern of check-in and check-out bike activity between two stations is similar. Hence, I will assign them into the same cluster.

Complete linkage

Complete linkage is applied to calculate the distance between two clusters \(A\) and \(B\). I prefer compact small diameter cluster over stringy ones.

\[ d(A,B)=max(d_{cos}(x,y):x \in A, y\in B) \]

Choose proper k

In agglomerative hierarchical clustering, I need to define the number of clusters in the clustering algorithm. From the elbow plot, I select \(k=3\).

Hierarchical clustering result

The clustering result is displayed below.

Three different patterns in the temporal pick and return activities are distinguished. From the line chart for each group, I can label the groups according to their daily check-in and check-out patterns.

• 100 stations are assigned to group 1. These stations show a slightly higher level of both check-in and check-out activity from 11:00 to 15:00 and 21:00 to 24:00. There is no significant difference between check-in and check-out activity. I can call group 1 an AVG group.

• 35 stations are assigned to group 2. These stations show a much more active check-out rate in the morning, from 11:00 to 15:00, while the check-in rate is about two times higher than the check-out one during night, from 20:00 to 23:00. I can call group 2 a Morning-Out-Night-In group.

• 37 stations are assigned to group 3. These stations share the opposite pattern compared to the Morning-Out-Night-In group. Also, this group show an active dynamism from noon to midnight. Moreover, the rate of check-in bike reached the peak (about 200) at 13:00. On the other hand, the rate of check-out bike hit about 175 at 21:00. I can call group 3 a Morning-In-Night-Out group.

When I look across the three groups, I can conclude that in the morning AVG and MONI stations are major origins of check-out while MINO stations are major destinations of check-in However, in the night, the situation is totally opposite. MINO stations are major origins of check-out while AVG and MONI stations are major destinations of check-in.

I can also use rose diagram to display the pattern of daily check-in and check-out bike flow between three clusters. The height of the bar is proportional to the trip duration in each timewindow. What interesting about the data is that trips starting from MINO stations are shorter than the other two. In AVG and MONI group, riders tend to have longer trips during evening.

I also use rose diagram of average distance in each timewindow. It should be noted that the average distance from AVG stabilizes at a high level in the morning and fluctuate slightly after noon. On the other hand, the one from MONI is short during the morning, but grows rapidly before noon and also remains at a high level in the night. The average distance from MONI flutters gently all day long.

I summarise some trip features for the three groups. From the table I know that AVG has the longest average trip duration, and AVG group seems to be most welcomed by casual riders. Take the large group size, it’s not surprised to see that AVG has the most trip records.

MONI group has the longest average trip distance. The member ratio is similar to the MINO one. The average trip duration is slightly smaller than the AVG one.

MINO has the shortest average trip distance, and it enjoys the highest member ration. It’s reasonable to infer that stations in MINO is more popular in member riders.

The geographical distribution of clusters is displayed below. I can use this plot to examine whether the temporal activities are related to the spatial factors. Station within the same cluster tend to locate in Toronto with certain patterns.

• AVG: Stations in AVG group is colored by red dots. These red dots spread all over the city.

• MONI: Station in MONI group is colored with green dots. MONI stations seem more likely to lie along the lake and lie around the Toronto downtown. The proportion of MONI stations amounts to nearly half in Toronto-St. Paul’s, and even exceeds half in Beaches-East York.

• MINO: Station in MINO group is colored with blue dots. MINO stations are more concentrated in the center of Toronto. These stations are only located in University-Rosedale, Spadina-Fort York, and Toronto-St. Paul’s. Since these three wards are main working and study areas, it’s reasonable to speculate that office workers and students are the major users of MINO.

Wards and clusters

I use glyphs to show the group status in each ward. Areas of circles are proportional to the number of stations. Pie slices proportional to station number in each group.

How I gather this dataset?

There are two major resources for this dataset.

One is from Bike Share Toronto Rideship Data. All the trip in this dataset is anonymized. Due to my computational limit, I only select 2016 Q4 to study. This dataset contains trip data, including:

• Trip start day and time
• Trip end day and time
• Trip duration
• Trip start station
• Trip end station
• User type

I download the dataset and use Excel to modify the date format.

screen shot of 2016 Q4 sharing bike trip

The information of station coordinates is found from publicbikesystem. I transform the .json file into .csv, and use Excel-vlookup to join the station location file with the trip file with key word station name.

Because of the development of sharing bike system in Toronto, location of some stations can’t be found nowadays. Thus, I just abandon trips from or to these stations. The total number of ride records drops from 217569 to 142866.

I also calculate the trip distance according to the coordinates with {r eval=F} distHaversine() function in {r eval=F} library(geosphere).

System Information

## R version 3.5.1 (2018-07-02)
## Platform: x86_64-apple-darwin15.6.0 (64-bit)
## Running under: macOS Sierra 10.12.6
## 
## Matrix products: default
## BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
## 
## locale:
## [1] C
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] ape_5.2                  factoextra_1.0.5        
##  [3] proxy_0.4-22             leaflet.minicharts_0.5.4
##  [5] leaflet.extras_1.0.0     bindrcpp_0.2.2          
##  [7] rgdal_1.3-6              sp_1.3-1                
##  [9] geosphere_1.5-7          ggplot2_3.1.0           
## [11] reshape2_1.4.3           dplyr_0.7.8             
## [13] leaflet_2.0.2            lubridate_1.7.4         
## 
## loaded via a namespace (and not attached):
##  [1] tidyselect_0.2.5  purrr_0.2.5       lattice_0.20-35  
##  [4] colorspace_1.3-2  htmltools_0.3.6   viridisLite_0.3.0
##  [7] yaml_2.2.0        rlang_0.3.0.1     ggpubr_0.2       
## [10] later_0.7.5       pillar_1.3.0      glue_1.3.0       
## [13] withr_2.1.2       bindr_0.1.1       plyr_1.8.4       
## [16] stringr_1.3.1     munsell_0.5.0     gtable_0.2.0     
## [19] htmlwidgets_1.3   codetools_0.2-15  evaluate_0.12    
## [22] labeling_0.3      knitr_1.20        httpuv_1.4.5     
## [25] crosstalk_1.0.0   parallel_3.5.1    highr_0.7        
## [28] Rcpp_1.0.0        xtable_1.8-3      promises_1.0.1   
## [31] scales_1.0.0      jsonlite_1.6      mime_0.6         
## [34] gridExtra_2.3     digest_0.6.18     stringi_1.2.4    
## [37] ggrepel_0.8.0     shiny_1.2.0       grid_3.5.1       
## [40] tools_3.5.1       magrittr_1.5      lazyeval_0.2.1   
## [43] tibble_1.4.2      cluster_2.0.7-1   crayon_1.3.4     
## [46] tidyr_0.8.2       pkgconfig_2.0.2   assertthat_0.2.0 
## [49] rmarkdown_1.11    viridis_0.5.1     R6_2.3.0         
## [52] nlme_3.1-137      compiler_3.5.1