Understanding distance arg in Pandana.aggregate() with UrbanAccess network

[kuan]

I am posting here to see if I can gain a better understanding of the aggregate function in Pandana. The first positional argument in the function is distance.

When one uses UrbanAccess to create the network, we calculate time in lieu of distance as the measure/cost along a given edge. This value is set as the "weight" column in the resulting urbanaccess_nw.net_connector_edges dataframe.

Ques. 1: Is that correct? Is/does the "weight" column in urbanaccess_nw.net_connector_edges represent the time value (in minutes) along a given edge?

In Pandana, the net_aggregate function allows us to perform, for example, sums within a given "distance." This distance (the first positional argument in the net_aggregate function), in the case of the UrbanAccess output network, is time. Thus, if we wanted, say, jobs within 15 minutes (and had set a dataset that held employment counts), we could run the following operation:

p_net.aggregate(15, type="sum", decay="linear", name="emp")

where p_net is an object instantiated from the pdna.Network class.

Ques. 2: Is the above statement true? aggregate seems to be returning the same result, regardless of the integer value being supplied for the distance argument.

[blanchard]

Hi Kuan,
Great questions! To answer your questions:

1. Yes the default edge weight column generated using UrbanAccess is in units of time (minutes).

2. Yes the distance parameter in aggregate should be specified in the impedance unit that is inside of your specified weight column. So in this case it would be time (minutes). It is understandable that the name of the parameter distance in aggregate could be more clear and could be instead changed to impedance. In the latest release of Pandana the docstrings have been updated for the distance parameter to specifically clear this issue up as we figured this may be confusing, however in a future release we can consider changing the parameter name itself to be more clear.

In terms of the issue you are seeing, from the bit of code you provided it looks like the pandana network object you created is using only the urbanaccess_nw.net_connector_edges component of the urbanaccess network. Instead you should use the combined network that holds the weight of not only the connector edges but also the transit edges and the pedestrian edges so you should use this: urbanaccess_nw.net_edges. Your urbanaccess_nw.net_connector_edges only represent the connection between the transit and pedestrian networks and understandably these travel times would be very small with the exception of connections with large headways if you specified that in the network creation process.

We will be releasing a demo for urbanaccess in the coming weeks that should make this workflow between urbanaccess and pandana more clear.

I hope this clears up this issue please post here if this helps clear up the issue or if you have any questions about this.

[kuan]

Update:

I spoke to soon. I miscopied my code when writing the prior question. I did indeed incorrectly utilize urbanaccess_nw.net_connector_edges, but had done so for a prior step, when calculating a color_range variable for plotting the UA network.

I want to avoid dumping too much code here, but wanted to sketch out my methodology. And show the steps taken post setup of UA network (transit + OSM).

First, I get the various Series variables needed to instantiate a new Pandana object.

# insantiate a pandana network object
p_net = pdna.Network(
                     ua_nw.net_nodes.x,
                     ua_nw.net_nodes.y,
                     ua_nw.net_edges.from, 
                     ua_nw.net_edges.to, 
                     ua_nw.net_edges[['weight']])

At this point I could precompute a horizon distance (edited out of this comment). Afterwards, I load in a dataset that contains some metadata I want to measure with the Pandana network graph.

# read in an example dataset as a geodataframe
df, geometry = _create_geodataframe_from_csv('./data/blocks.csv')
gdf = gpd.GeoDataFrame(df, crs={'init': 'epsg:4326'}, geometry=geometry)

# set the Pandana network with this new dataset
p_net.set(p_net.get_node_ids(gdf.x, gdf.y), variable=gdf.emp, name='employment')

At this point, I believe I should be able to run the aggregate function and get, for example, sums within a given distance (or, in this case, minute measure). Here's an example of that being run, where n is some "distance" limit:

s = p_net.aggregate(n, type='sum', decay='linear', name='emp')

Ques.: The above function returns a Series that is the exact same, no matter what value for n is provided. In the loaded in dataset, there are ~7,000 rows that are associated with ~3,500 nodes in Pandana. In the UA network, the min weight value is 0.00 and the max weight value is 50.82. Making n any value between 0 and 50 results in the same Series. Making the n value 0, results in a Series of 0 values.

[blanchard]

No problem without seeing all your steps leading up to this result it will be hard to diagnose the problem. Could you send me the full urbanaccess script that creates the network (start to finish) and then the full script attempting the pandana query with annotations. Including a download link to your original saved network files and your block.csv file will also help.

[kuan]

Thanks Sam

I've made an example repo (perhaps might be handy as well in future if it is not entirely incorrect, to be used for a walkthrough of some sort).

I am in the process of cleaning the data (blocks.csv) and should have that up in the next few minutes.

[blanchard]

Thank you for the script and data Kuan!

Ok I have found a couple of issues in the script that are causing the issues:

1. When you create the pandana Network object you should specify that the network is twoway=False. You have left the default which is the opposite: twoway=True. UrbanAccess generates a explicit oneway network with edges that represent one direction of travel.

2. Pandana currently requires from and to ids to be integer types - this is a documented issue on the repo. Because of this UrbanAccess generates two types of ids: a) from and to ids that are human readable strings and that cannot at this time be used in pandana; and b) pandana readable ids from_int and to_int. You should use the from_int and to_int in your urbanaccess_nw.net_edges when building your pandana Network. This will address the issue you are seeing in your results.

3. I also noticed your pandana plot bboxis in the wrong tuple format for pandana's plotbbox parameter. UrbanAccess and Pandana use different bbox tuple coordinate pair ordering which is understandably confusing - something we can address at another time, however for now the docstrings should indicate the correct format. This is why your plots are not showing up correctly.

These issues all point to the need for urbanaccess to have a full demo on the repo which will be complete in the coming weeks.

My results for 15 and 60 min travel time query are below:

15 min:
max=28499.5
mean=2745.57275391
min=0.0

60 min:
max=131889.0
mean=49924.8945312
min=0.0

I hope this clears up the issue please post back if it does or if you have any other questions.

[kuan]

Amazingly helpful - thanks so much!

[kuan]

Wanted to share my results here as well - buckets are working smoothly now, thanks again for your help!

Tweeted animated GIF: https://twitter.com/buttsmeister/status/857344074189119488

Or static image of 60 min bucket:

[stefancoe]

Hi, I just started working with UrbanAccess and had the following code working earlier today. I started cleaning it up and then of course it stopped working. I figured I'd post here rather than start a new thread. Hope that is ok. It crashes on the last line and gives me this message:

[error [error src/contraction_hierarchies/src/libch.cpp:368[error ] src/contraction_hierarchies/src/libch.cpp:368POI Category is out of Bounds]
An internal error has occurred in the Interactive window. Please restart Visual Studio.
POI Category is out of Bounds[error src/contraction_hierarchies/src/libch.cpp:[error src/contraction_hierarchies/src/libch.cpp:368] POI Category is out of Bounds
The Python REPL process has exited

Can you see anything wrong? I saved the pandana network from when it was working and it still works if i load that network back in. The GTFS data can be found here:
https://file.ac/-LhHHOc0HaA/

import urbanaccess as ua
import pandas as pd
import pandana as pdna 

# GTFS
gtfs_df = ua.gtfs.load.gtfsfeed_to_df(r'D:\stefan\Isochrone\repository\gtfs', 
                                      validation=False, verbose=True, 
                                      bbox=(-122.5411,47.0845,-121.0657,47.7803), 
                                      remove_stops_outsidebbox=True, 
                                      append_definitions=False)


ua_net = ua.gtfs.network.create_transit_net(gtfs_df, 
                                   day='monday', 
                                   timerange=["06:00:00","09:00:00"], 
                                   overwrite_existing_stop_times_int=False, 
                                   use_existing_stop_times_int=True, 
                                   save_processed_gtfs=False, 
                                   save_dir='data', 
                                   save_filename=None)

ua.gtfs.headways.headways(gtfs_df,["06:00:00","09:00:00"])

# OSM:
osm_data = ua.osm.load.ua_network_from_bbox(bbox=(-122.5411,47.0845,-121.0657,47.7803), 
                                           network_type='walk', 
                                           timeout=180, 
                                           memory=None, 
                                           max_query_area_size=2500000000L, 
                                           remove_lcn=True)

ua_net = ua.osm.network.create_osm_net(osm_edges=osm_data[1], osm_nodes=osm_data[0], travel_speed_mph=3, network_type='walk')

# Integrate the network
ua_integrated_net = ua.network.integrate_network(ua_net, 
                             headways=True, 
                             urbanaccess_gtfsfeeds_df=gtfs_df, 
                             headway_statistic='mean')

ua.network._format_pandana_edges_nodes(ua_integrated_net.net_edges, ua_integrated_net.net_nodes)

# Create a pandana network
imp = pd.DataFrame(ua_integrated_net.net_edges['weight'])
pdna_net = pdna.network.Network(ua_integrated_net.net_nodes.x, ua_integrated_net.net_nodes.y, 
                                ua_integrated_net.net_edges.from_int, ua_integrated_net.net_edges.to_int, imp, False)

#pdna_net.save_hdf5('D:/stefan/Isochrone/repository/UrbanAccess/pdna_integrated_net.h5') 
#pdna_net2 = pdna.Network.from_hdf5('D:/stefan/Isochrone/repository/UrbanAccess/pdna_integrated_net.h5')

dist = 30
pdna_net.init_pois(1, dist, 1)
#-122.337999,47.613309,-122.336969,47.614235
x = pd.Series(-122.355)
y = pd.Series(47.687)
#Set as a point of interest on the pandana network
pdna_net.set_pois('tstop', x, y)

    #Find distance to point from all nodes, everything over max_dist gets a value of 99999
res = pdna_net.nearest_pois(dist, 'tstop', num_pois=1, max_distance=99999)

Thanks!

-Stefan

[kuan]

Hi Stefan

Sorry for the delay I just noticed this. Might be better to create a new subject so others notice it since it does end up being a little 'below the fold' and thus maybe not as 'findable.'

The contract hierarchies error your are seeing is occurring (I believe, I just did a cursory visual review of your code and did not implement it myself) in this line: pdna_net.init_pois(1, dist, 1)

The error was triggered, specifically, by this assertion (line 368): https://github.com/UDST/pandana/blob/master/src/contraction_hierarchies/src/libch.cpp#L368

Contraction hierarchies precompute distances within an distance horizon from a given node to others that are 'within range.' This is handy because it has the potential to speed up the analysis processes significantly. It looks like the data that you are putting into that step is out of bounds from the given bounding box.

I haven't worked with POIs but perhaps the issue is related to the fact that you are initializing POIs before you set them (which in your code occurs ~5 lines later).

[stefancoe]

Thanks Kuan. The error I get occurs on the last line where I am calling .nearest_pois. Also, the init_pois has to be called before set_pois. The interesting thing is that this code works when using the pandana network that I saved to disk when the script was working so I suspect it is a network issue. Also, I have verified that the set of coordinates are inside the bounds of of the pandana network. I will create a new thread here or as an issue in the UrbanAccess issue tracker. Thanks again!

[blanchard]

Stefan

Thank you for your post Stefan, we will take a look at this later this week and get back to you via your issue on the GitHub repo.

[blanchard]

Wanted to follow up on this post. We have just created a demo for UrbanAccess that shows you an entire workflow in a jupyter notebook that can be accessed here.