Use of RFID technology to characterize feeder visitations and contact network of hummingbirds in urban habitats.
Ruta R. Bandivadekar, Pranav S. Pandit, Rahel Sollmann, Michael J. Thomas, Scott Logan, Jennifer Brown, A.P. Klimley, and Lisa A. Tell
The notebook demonstrates analysis presented in the manuscript
We developed a Python package 'taggit' for reading the tag detection data collapsing it into visits and visualization of PIT tagged hummigbird activity at the novel feeding stations.
## setting up local path and importing python packagesdata_path='C:/Users/Falco/Desktop/directory/taggit/data/'path='C:/Users/Falco/Desktop/directory/taggit'output_path='C:/Users/Falco/Desktop/directory/taggit/outputs'## importing python packagesimportosasosimportpandasaspd%matplotlibinlinefrombokeh.ioimportshow, output_file , output_notebook, saveoutput_notebook()
frombokeh.modelsimportPlot, Range1d, MultiLine, Circle, HoverTool, TapTool, BoxSelectTool, BoxZoomTool, ResetTool, PanTool, WheelZoomTool, graphsfrombokeh.palettesimportSpectral4frommatplotlibimportpyplotaspltos.chdir(path)
fromscipyimportstatsimportnumpyasnpfromscipyimportstatsimportjoypyimportseabornassnsfrommatplotlibimportpyplotaspltimportnumpyasnpfrommatplotlibimportcm# Importing the taggit packagefromtaggitimportfunctionsasHTfromtaggitimportinteractionsasHxnetimportmatplotlib.styleasstylestyle.use('fivethirtyeight')
plt.rcParams['lines.linewidth'] =1dpi=1000
1.read all text files from a folder (just keep tag files in this folder no other txt files)
2.remove unwanted tag reads: fishtags, other removes which were some mistakes done during installing readers
3.ability to restrict analysis to to specific location/readers this function will filter the data accordingly
4.for the current analysis presented in the paper we are not inlcuding rufous humminbirds
Beacuse of battery and power problems A4 reader was not function for some time. We use data from A2 reader which for a top reader from the DAFS for that duration
C:\Users\Falco\Anaconda2\lib\site-packages\ipykernel_launcher.py:3: DeprecationWarning:
.ix is deprecated. Please use
.loc for label based indexing or
.iloc for positional indexing
See the documentation here:
http://pandas.pydata.org/pandas-docs/stable/indexing.html#ix-indexer-is-deprecated
This is separate from the ipykernel package so we can avoid doing imports until
print ('number of tag reads during study period= '+str(tag_data.shape[0]))
number of tag reads during study period= 118234
in the manuscript we are analysing data only until March 2018
tag_data=tag_data.ix[:'2018-03-31']
C:\Users\Falco\Anaconda2\lib\site-packages\ipykernel_launcher.py:1: DeprecationWarning:
.ix is deprecated. Please use
.loc for label based indexing or
.iloc for positional indexing
See the documentation here:
http://pandas.pydata.org/pandas-docs/stable/indexing.html#ix-indexer-is-deprecated
"""Entry point for launching an IPython kernel.
even though we have 155 birds detected in the data, we restrict our analysis to birds that have been tagged until end February 2018
collapse_reads_10_seconds
this function will
1. for each individual bird, it will calculate different between each reads
2. if the difference is more that 0.11 seconds it will identify it as a new visit.
3. for each new visit, it will identify the starting time, ending time
print ('number of visits recorded during study period= '+str(visit_data.shape[0]))
number of visits recorded during study period= 65476
read metadata file
this function will
1. read the master metadata file for all the birds PIT tagged until now
2. filter the data according to the study location
meta=HT.read_metadata(data_path=data_path , filename='PIT tagged Birds Info_HHCP_For manuscript_04_16_2018.xlsx',
restrict=True)
meta=meta.drop_duplicates('Tag Hex', keep='first')
meta['Location Id'].unique()
## removing data for birds tagged in another site not in this studyremove_glide=list(set(meta[meta['Location Id'] =='GR']['Tag Hex'].unique().tolist()).difference(set(visit_data.ID.unique().tolist())))
meta=meta[~meta['Tag Hex'].isin(remove_glide)]
meta.Species.unique()
meta.shape
this function will
1. merge the tag read data with the metadata using the 'Tag Hex' column
2. Merge the data only after running collapse_reads_10_seconds function
3. identifies birds which were tagged and found on the same day in 2016
3D6.00184967A3 date same, first recorded 2016-09-23 08:56:09 tagged on 2016-09-23
3D6.00184967AE date same, first recorded 2016-09-09 10:23:19 tagged on 2016-09-09
3D6.00184967B5 date same, first recorded 2016-09-23 08:54:49 tagged on 2016-09-23
3D6.00184967C6 date same, first recorded 2016-09-09 09:55:46 tagged on 2016-09-09
3D6.00184967D3 date same, first recorded 2016-09-23 08:52:58 tagged on 2016-09-23
3D6.00184967D6 date same, first recorded 2016-09-23 08:05:57 tagged on 2016-09-23
3D6.00184967D7 date same, first recorded 2016-09-23 08:54:00 tagged on 2016-09-23
3D6.00184967DB date same, first recorded 2016-10-21 08:20:40 tagged on 2016-10-21
even though we have 155 birds detected in the data, we restrict our analysis to birds that have been tagged until end February 2018
Daily visitations of PIT- tagged male and female Anna’s Hummingbirds at the feeding stations at Sites 1 and 2 in Northern California and Site 3 in Southern California from September 2016 to March 2018.
importmatplotlib.tickerastickerimportmatplotlib.datesasmdatesdata.Sex.fillna('unknown', inplace=True)
alhu_data=data[data.Species=="ALHU"]
anhu_data=data[data.Species=="ANHU"]
plt.rcParams['font.size'] =13plt.rcParams['font.family'] ='Times New Roman'plt.rcParams['axes.labelsize'] =plt.rcParams['font.size']
plt.rcParams['axes.titlesize'] =1.5*plt.rcParams['font.size']
plt.rcParams['legend.fontsize'] =plt.rcParams['font.size']+1plt.rcParams['xtick.labelsize'] =plt.rcParams['font.size']
plt.rcParams['ytick.labelsize'] =plt.rcParams['font.size']
defplotvisits(timeunit, data, ax, c= ['#e41a1c', '#377eb8', '#4daf4a'], style='-'):
"""timeunite: 10Min, D = day, W = week, M = month"""iftimeunit=='10Min':
t='10 minutes'eliftimeunit=='D':
t='day'eliftimeunit=='W':
t='week'iftimeunit=='M':
t='month'a=data.groupby([ pd.Grouper(freq=timeunit), 'Sex'])['ID'].count().unstack('Sex')
a.plot(kind="line", style=style, rot=45, ax=ax,stacked=False, color=c,)
#set ticks every monthax.xaxis.set_major_locator(mdates.MonthLocator())
#set major ticks formatax.xaxis.set_major_formatter(mdates.DateFormatter('%b %Y'))
## hiding every second monthforindex, labelinenumerate(ax.xaxis.get_ticklabels()):
ifindex%3!=0:
label.set_visible(False)
#ax.set_title('visits per '+t)ax.set_xlabel('Time')
ax.set_ylabel('Number of visits per day')
f, (ax1) =plt.subplots(1, 1, figsize=(8,6))
plotvisits(timeunit='D', data=anhu_data, ax=ax1)
f.tight_layout(rect=[0, 0.03, 1, 0.95])
plt.savefig(output_path+'\Figure4.png', dpi=dpi)
plt.savefig(output_path+'\Figure4.eps',format='eps', dpi=dpi)
plt.show()
Tagging activity Figure 5 Inset: The cumulative tagging effort throughout the study.
importmatplotlib.datesasmdatesfig, ax1=plt.subplots(1, figsize=(6,4))
idx=pd.date_range('06.01.2016', '02.28.2018')
a=meta.groupby('Tagged Date')['Tag Hex'].nunique()
a.index=pd.DatetimeIndex(a.index)
a=a.reindex(idx, fill_value=0)
#tl = a.cumsum()#tl.columns = ['sampling']#tl.plot.line(drawstyle = 'steps', label='Animals', rot =45,ax = ax1)a.resample('D').cumsum().plot(kind="line", rot=45,ax=ax1)
#set ticks every month#ax1.xaxis.set_major_locator(mdates.MonthLocator())#set major ticks format#ax1.xaxis.set_major_formatter(mdates.DateFormatter('%Y'))## hiding every second month#for index, label in enumerate(ax1.xaxis.get_ticklabels()):# if index % 3 != 0:# label.set_visible(False)ax1.set_ylabel('Individuals tagged', fontsize=11)
ax1.set_xlabel('Time', fontsize=11)
fig.tight_layout(rect=[0, 0.03, 1, 0.95])
plt.savefig(output_path+'/'+'cumulative_tagging.png', dpi=dpi)
plt.savefig(output_path+'/'+'cumulative_tagging.eps',format='eps', dpi=dpi)
plt.show()
C:\Users\Falco\Anaconda2\lib\site-packages\ipykernel_launcher.py:10: FutureWarning:
.resample() is now a deferred operation
You called cumsum(...) on this deferred object which materialized it into a series
by implicitly taking the mean. Use .resample(...).mean() instead
# Remove the CWD from sys.path while we load stuff.
Figure 6:
Comparison of the daily detections from the two different antennas at the double antenna feeding station transceiver at Site 2 over time. The top antenna for this feeding station was deployed in May 2017 and discontinued in January 2018 for data collection. Overall, the total number of visits detected by the side antenna exceeded the number of visits for the top antenna but not enough to warrant the addition of a second antenna. Both antennas detected the presence of the same number of individual birds.
C:\Users\Falco\Anaconda2\lib\site-packages\ipykernel_launcher.py:11: DeprecationWarning:
.ix is deprecated. Please use
.loc for label based indexing or
.iloc for positional indexing
See the documentation here:
http://pandas.pydata.org/pandas-docs/stable/indexing.html#ix-indexer-is-deprecated
# This is added back by InteractiveShellApp.init_path()
C:\Users\Falco\Anaconda2\lib\site-packages\ipykernel_launcher.py:12: DeprecationWarning:
.ix is deprecated. Please use
.loc for label based indexing or
.iloc for positional indexing
See the documentation here:
http://pandas.pydata.org/pandas-docs/stable/indexing.html#ix-indexer-is-deprecated
if sys.path[0] == '':
print ('side antenna recorded '+str(dataA4.shape[0]) +' visits')
side antenna recorded 7825 visits
print ('top antenna recorded '+str(dataA2.shape[0]) +' visits')
top antenna recorded 7620 visits
print ('side antenna recorded '+str(len(dataA4.ID.unique())) +' PIT tagged birds')
side antenna recorded 20 PIT tagged birds
print ('top antenna recorded '+str(len(dataA2.ID.unique())) +' PIT tagged birds')
top antenna recorded 20 PIT tagged birds
Figure 6
dpi=1000
defplotvisits(timeunit, data, ax, c= ['#e41a1c', '#377eb8', '#4daf4a']):
"""timeunite: 10Min, D = day, W = week, M = month"""iftimeunit=='10Min':
t='10 minutes'eliftimeunit=='D':
t='day'eliftimeunit=='W':
t='week'iftimeunit=='M':
t='month'data.groupby([ pd.Grouper(freq=timeunit)])['ID'].count().plot(kind="line", rot=45, ax=ax)
#set ticks every month#ax.xaxis.set_major_locator(mdates.MonthLocator())#set major ticks format#ax.xaxis.set_major_formatter(mdates.DateFormatter('%Y %B'))## hiding every second month#for index, label in enumerate(ax.xaxis.get_ticklabels()):# if index % 3 != 0:# label.set_visible(False)#ax.set_title('visits per '+t)ax.set_xlabel('Time')
ax.set_ylabel('Number of visits per day')
f, (ax1) =plt.subplots(1, 1, figsize=(8,6))
plotvisits(timeunit='D', data=dataA4, ax=ax1)
plotvisits(timeunit='D', data=dataA2, ax=ax1)
labels= ['Jun 2017', 'Jul 2017', 'Aug 2017','Sep 2017', 'Oct 2017', 'Nov 2017', 'Dec 2017', 'Jan 2018', 'Feb 2018']
ax1.set_xticklabels(labels)
#set ticks every month#ax1.xaxis.set_major_locator(mdates.MonthLocator())#set major ticks format#ax1.xaxis.set_major_formatter(mdates.DateFormatter('%Y %B'))## hiding every second month#for index, label in enumerate(ax1.xaxis.get_ticklabels()):# if index % 3 != 0:# label.set_visible(False)f.legend(['side antenna reader unit', 'top antenna reader unit'],loc='best')
#f.suptitle('Hummingbird visits to feeders', fontsize = 24)f.tight_layout()#rect=[0, 0.03, 1, 0.95]f.savefig(output_path+'/Figure 5.png', dpi=dpi)
f.savefig(output_path+'/'+'Figure 5.eps',format='eps', dpi=dpi)
plt.show()
C:\Users\Falco\Anaconda2\lib\site-packages\matplotlib\legend.py:649: UserWarning: Automatic legend placement (loc="best") not implemented for figure legend. Falling back on "upper right".
warnings.warn('Automatic legend placement (loc="best") not '
Gender association with activity of bird visitation metrices
C:\Users\Falco\Anaconda2\lib\site-packages\scipy\stats\stats.py:1706: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
return np.add.reduce(sorted[indexer] * weights, axis=axis) / sumval
data.visit_duration.describe()
count 59050
mean 0 days 00:00:07.291244
std 0 days 00:00:17.425316
min 0 days 00:00:00
25% 0 days 00:00:00
50% 0 days 00:00:00
75% 0 days 00:00:10
max 0 days 00:10:15
Name: visit_duration, dtype: object
plt.hist(Bird_summary.duration_total)
plt.show()
Bird_summary.duration_total.describe()
count 141.000000
mean 3053.531915
std 8455.482026
min 0.000000
25% 0.000000
50% 0.000000
75% 1331.000000
max 59922.000000
Name: duration_total, dtype: float64
C:\Users\Falco\Anaconda2\lib\site-packages\ipykernel_launcher.py:1: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
"""Entry point for launching an IPython kernel.
count 91.000000
mean 0.000651
std 0.001232
min 0.000000
25% 0.000000
50% 0.000000
75% 0.001028
max 0.007655
Name: proportion_present, dtype: float64
f_pt.proportion_present.describe()
count 50.000000
mean 0.000731
std 0.001580
min 0.000000
25% 0.000000
50% 0.000006
75% 0.000628
max 0.007378
Name: proportion_present, dtype: float64
Bird_summary.proportion_present.describe()
count 141.000000
mean 0.000680
std 0.001361
min 0.000000
25% 0.000000
50% 0.000000
75% 0.000874
max 0.007655
Name: proportion_present, dtype: float64
Diel activity of Humminbirds
Figure 7:
Diel plots of mean hourly hummingbird visits to the feeding stations at three sites at northern and southern California from September 2016 to March 2018. The bar denotes the mean hourly visits by hummingbirds and black lines show the standard error. All distributions were statistically non-uniform (probability values reported by season).
C:\Users\Falco\Anaconda2\lib\site-packages\matplotlib\projections\polar.py:58: RuntimeWarning: invalid value encountered in less
mask = r < 0
Rayleigh test identify a non-uniform distribution, i.e. it is
designed for detecting an unimodal deviation from uniformity
winter: p = 0.000
spring: p = 0.000
summer: p = 0.000
fall: p = 0.000
Wall time: 24.3 s
Rayleigh test identify a non-uniform distribution, i.e. it is
designed for detecting an unimodal deviation from uniformity
winter: p = 0.000
spring: p = 0.000
summer: p = 0.000
fall: p = 0.000
Wall time: 18.9 s
Rayleigh test identify a non-uniform distribution, i.e. it is
designed for detecting an unimodal deviation from uniformity
winter: p = 0.000
spring: p = 0.000
summer: p = 0.000
fall: p = 0.000
Wall time: 12.6 s
Rayleigh test identify a non-uniform distribution, i.e. it is
designed for detecting an unimodal deviation from uniformity
winter: p = 0.000
spring: p = 0.000
summer: p = 0.000
fall: p = 0.000
Wall time: 14.6 s
Rayleigh test identify a non-uniform distribution, i.e. it is
designed for detecting an unimodal deviation from uniformity
winter: p = 0.000
spring: p = 0.000
summer: p = 1.000
fall: p = 0.007
Wall time: 11.6 s
Rayleigh test identify a non-uniform distribution, i.e. it is
designed for detecting an unimodal deviation from uniformity
winter: p = 0.000
spring: p = 0.000
summer: p = 1.000
fall: p = 0.007
Wall time: 10.9 s
Rayleigh test identify a non-uniform distribution, i.e. it is
designed for detecting an unimodal deviation from uniformity
winter: p = 0.357
spring: p = 0.692
summer: p = 1.000
fall: p = 0.518
Wall time: 10.3 s
bird_summaries
This function:
calculates summaries for each tagged bird
save a csv file with summary output
columns are
Tag Hex,Species,Sex, Age, tagged_date, first_obs_aft_tag, date_min, date_max, obser_period, date_u, Location
C:\Users\Falco\Anaconda2\lib\site-packages\ipykernel_launcher.py:3: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
This is separate from the ipykernel package so we can avoid doing imports until
ID date
3D6.00184967A4 2018-01-04 3
2018-01-06 1
3D6.00184967A7 2017-06-19 1
3D6.00184967AD 2017-05-22 2
2017-05-23 11
2017-05-24 7
2017-05-25 4
3D6.00184967AF 2017-05-23 2
2017-05-24 3
2017-05-25 1
2018-01-05 3
2018-01-06 1
3D6.00184967BC 2017-06-19 1
3D6.00184967BF 2018-01-04 1
2018-02-10 1
3D6.1D593D7848 2018-02-13 1
Name: visit_start, dtype: int64
Activity of an after hatch year male Anna’s hummingbird with a subcutaneously placed PIT at Site 2 between May 22nd and May 26th 2017. This bird was detected at the feeder after sunset, evident in the peaks of hourly visits during the night (green shaded sections of plot).
Activity of PIT tagged Anna’s Hummingbirds (Calypte anna) detected between 10 PM to 4 AM for the study period (September 2016- March 2018). This nocturnal activity was only seen at study site 2 in northern California. The shaded portion represents night time. Days preceding and succeeding the night
The preference of individual Anna’s and Allen’s Hummingbirds for the primary, secondary, and tertiary feeders in Site 2 in Northern California. Solid circles show the data points.
Figure 10: (Manuscript has a figure generated using Gephi)
Contact network of tagged Anna’s (pink nodes) and Allen’s hummingbirds (green nodes). The sex of the hummingbirds indicated by M for males and F for females. The size of the node is proportional to the degree of the interaction. Edges represent time spent together at the feeding station and the width of the edge width is proportional to the time spent together.
color_map= {"ANHU":'#e41a1c', "ALHU":'#377eb8'}
plt.figure(figsize=(25,25))
options= {
'edge_color': '#999994',
'font_weight': 'regular',
'label': True,
'alpha': 0.8
}
colors= [color_map[Hnet.node[node]['Species']] fornodeinHnet]
#sizes = [G.node[node]['nodesize']*10 for node in G]w=nx.get_edge_attributes(Hnet,'weight').values()
w2= [x-1forxinw]
s= [1000+(10000*np.log2(x+1))+(np.log2(np.log2(x+1)+1)*10000) forxinnx.betweenness_centrality(Hnet,).values()]
l=nx.get_node_attributes(Hnet, 'Sex').values()
mapping=nx.get_node_attributes(Hnet, 'Sex')
#Hnet=nx.relabel_nodes(Hnet,mapping)"""Using the spring layout : - k controls the distance between the nodes and varies between 0 and 1- iterations is the number of times simulated annealing is rundefault k=0.1 and iterations=50"""positions=forceatlas2.forceatlas2_networkx_layout(Hnet, pos=None, iterations=2000)
nx.draw(Hnet, node_color=colors, node_size=s, pos=positions, width=w, **options)
nx.draw_networkx_labels(Hnet,positions,mapping,font_size=24)
ax=plt.gca()
ax.collections[0].set_edgecolor("#555555")
plt.savefig(output_path+'/Network_diagram.png', dpi=dpi)
plt.savefig(output_path+'/Network_diagram.eps', format='eps', dpi=dpi)
plt.show()
100%|████████████████████████████████████████████████████████████████████████████| 2000/2000 [00:00<00:00, 2148.23it/s]
('BarnesHut Approximation', ' took ', '0.20', ' seconds')
('Repulsion forces', ' took ', '0.55', ' seconds')
('Gravitational forces', ' took ', '0.01', ' seconds')
('Attraction forces', ' took ', '0.06', ' seconds')
('AdjustSpeedAndApplyForces step', ' took ', '0.05', ' seconds')
Analysis for comparing degree distrbutions
Figure S2:
Results of permutation-based regression analysis to understand the effect of age and sex on the degree of in the observed network. Blue lines show the distribution of coefficients after 10,000 permutations. Red lines show original coefficients.
Results of permutation-based regression analysis to understand the effect of age and sex on the betweenness centrality of in the observed network. Blue lines show the distribution of coefficients after 10,000 permutations. Red lines show original coefficients.
Contact network of tagged Anna’s (pink nodes) and Allen’s hummingbirds (green nodes). The sex of the hummingbirds indicated by M for males and F for females. The size of the node is proportional to the degree of the interaction. Edges represent time spent together at the feeding station and the width of the edge width is proportional to the time spent together.