API Extravaganza! Combining Google Analytics & ORCID APIs - 22 Sept 2016

Video thumbnail (Frame 0) Video thumbnail (Frame 1197) Video thumbnail (Frame 2832) Video thumbnail (Frame 4246) Video thumbnail (Frame 7904) Video thumbnail (Frame 12175) Video thumbnail (Frame 13518) Video thumbnail (Frame 16326) Video thumbnail (Frame 18203) Video thumbnail (Frame 19623) Video thumbnail (Frame 20828) Video thumbnail (Frame 23448) Video thumbnail (Frame 25558) Video thumbnail (Frame 27745) Video thumbnail (Frame 28713) Video thumbnail (Frame 31749) Video thumbnail (Frame 34124) Video thumbnail (Frame 35206) Video thumbnail (Frame 36561) Video thumbnail (Frame 39177) Video thumbnail (Frame 40551) Video thumbnail (Frame 42393) Video thumbnail (Frame 43495) Video thumbnail (Frame 44568) Video thumbnail (Frame 45682) Video thumbnail (Frame 47560) Video thumbnail (Frame 48933) Video thumbnail (Frame 51423) Video thumbnail (Frame 52383) Video thumbnail (Frame 55978) Video thumbnail (Frame 57118) Video thumbnail (Frame 58744) Video thumbnail (Frame 59923) Video thumbnail (Frame 61221) Video thumbnail (Frame 64031) Video thumbnail (Frame 66168) Video thumbnail (Frame 67218) Video thumbnail (Frame 68497) Video thumbnail (Frame 69458) Video thumbnail (Frame 74087)
Video in TIB AV-Portal: API Extravaganza! Combining Google Analytics & ORCID APIs - 22 Sept 2016

Formal Metadata

API Extravaganza! Combining Google Analytics & ORCID APIs - 22 Sept 2016
Title of Series
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
This is a technical presentation that provides an introduction to the Google's APIs for (Google Analytics, Drive, and Sheets) APIs and how they can be combined with ORCID API and other external sources. More Specifically, we talk about: - Accessing Google APIs with a service account - Querying the Google Analytics Core Reporting API - Creating/managing files with Google Drive and Sheets APIs - Integrating data from externals sources using the ORCID public API Presenter: Liz Krznarich from ORCID Liz is a software developer and UI designer at ORCID, where she takes on projects ranging from server automation to user interface design, and development for content management systems. Liz has a range of experience in design, web development, technology training and academic libraries, and previously worked at the University of Wisconsin. Liz earned a BS in Graphic Design and a Master's in Library & Information Science from the University of Wisconsin.
Addition Goodness of fit Server (computing) Googol Multiplication sign Software developer Interface (computing) Software Google Analytics Mereology Wave packet
Building Structural load Software developer Google Analytics Cartesian coordinate system Perspective (visual) Neuroinformatik Digital photography Different (Kate Ryan album) Operator (mathematics) Software Video game Point cloud Energy level Quicksort Task (computing) Physical system Point cloud
Source code Facebook Multiplication Googol Structural load Different (Kate Ryan album) Structural load Single-precision floating-point format Source code Order (biology) Quicksort Cartesian coordinate system
Functional (mathematics) Game controller User interface Table (information) Combinational logic Set (mathematics) Rule of inference Spreadsheet Different (Kate Ryan album) Term (mathematics) Googol Computer programming Traffic reporting Computer architecture Task (computing) User interface Mobile app Interface (computing) Software developer Projective plane Computer Analytic set Bit Cartesian coordinate system Computer Googol Pi Personal digital assistant Computer programming Self-organization Quicksort
Web page Trail Service (economics) Identifiability Table (information) Link (knot theory) Demo (music) Materialization (paranormal) Spreadsheet Different (Kate Ryan album) Googol Query language Repository (publishing) File system Cuboid Website Extension (kinesiology) Proxy server Physical system Point cloud Trail Information Analytic set Ext functor Cartesian coordinate system Digital library Peer-to-peer Googol Spreadsheet Personal digital assistant Website Point cloud Quicksort Row (database)
Email Slide rule Asynchronous Transfer Mode Scripting language Inheritance (object-oriented programming) Installation art Table (information) Code Demo (music) Simultaneous localization and mapping Cellular automaton Visual system Google Analytics Parsing Directory service Mathematical analysis Dreizehn Virtual reality Googol Graph (mathematics) Query language Repository (publishing) Integrated development environment Website Game theory Default (computer science) Source code Execution unit Slide rule Sampling (statistics) Analytic set Range (statistics) Core dump Coma Berenices Digital object identifier CAN bus Event horizon Translation memory Website Right angle Queue (abstract data type) Key (cryptography) Quicksort Pulse (signal processing)
Trail Scripting language Link (knot theory) Demo (music) Google Analytics Mereology Theory Googol Repository (publishing) Codierung <Programmierung> Gamma function Normal (geometry) Formal grammar Data type User interface Source code Information Suite (music) Computer file Video tracking Sampling (statistics) Analytic set Ext functor Bit Data mining Data management Googol Repository (publishing) Personal digital assistant Website Physical system Pulse (signal processing) Design of experiments
Building View (database) Sine Programmable read-only memory Sheaf (mathematics) Analytic set Parameter (computer programming) Menu (computing) Revision control Category of being Moment of inertia Perpetual motion Query language Googol Query language Website Quicksort Gamma function
User interface Event horizon Query language Web page Query language Cloud computing Analytic set Website Gamma function Metric system Traffic reporting Dimensional analysis
Web page Digital filter Metric system View (database) Connectivity (graph theory) View (database) Dimensional analysis Analytic set Ext functor Google Analytics Limit (category theory) Electronic mailing list Dimensional analysis Sample (statistics) Personal digital assistant Query language Hausdorff dimension Metric system Traffic reporting
Filter <Stochastik> Trail Metric system Boolean algebra State diagram Visual system Maxima and minima Google Analytics Electronic mailing list Counting Event horizon Dimensional analysis Field (computer science) Subset Personal digital assistant Query language Gamma function Data type Home page Information Interactive television Dimensional analysis Total S.A. Arithmetic mean Query language Personal digital assistant String (computer science) GAUSS (software) Revision control Website Right angle Quicksort Metric system Window Traffic reporting
Link (knot theory) Twin prime Code Software developer Interactive television Analytic set Bit Laser Query language Sampling (music) Password Internet service provider Quicksort Gamma function Traffic reporting Library (computing) Resultant Sinc function
Mobile app Group action Link (knot theory) Software developer Software developer Projective plane Median Maxima and minima Coma Berenices Menu (computing) Flow separation 2 (number) Mathematics Googol Error message Different (Kate Ryan album) Googol Physical law Quicksort
Electronic data interchange Motif (narrative) Projective plane Sheaf (mathematics) Analytic set Mass Menu (computing) Dreizehn Uniform resource locator Error message Whiteboard Different (Kate Ryan album) Personal digital assistant Gamma function Library (computing) Wide area network
Scripting language Point (geometry) User interface Server (computing) Service (economics) GUI widget Multiplication sign Software developer Computer-generated imagery Projective plane Interactive television Virtual machine Cartesian coordinate system Type theory Googol Computer configuration Different (Kate Ryan album) Password Internet forum Natural number Authorization Quicksort Communications protocol
Domain name Service (economics) Computer file Demo (music) File format Server (computing) Demo (music) Projective plane Cartesian coordinate system Public-key cryptography Wave packet Different (Kate Ryan album) Password Acoustic shadow Quicksort Gamma function
Touchscreen Service (economics) Computer file Code Multiplication sign Demo (music) Division (mathematics) Median Graphic design Product (business) Time domain Different (Kate Ryan album) Googol Ring (mathematics) Authorization Gamma function Mutual information Dean number Server (computing) Analytic set Mathematics Process (computing) Error message Intrusion detection system Personal digital assistant Password Library (computing)
Source code Asynchronous Transfer Mode Information management Email Service (economics) Hoax Dependent and independent variables System administrator Demo (music) Web page Maxima and minima Cloud computing Analytic set Mass Arm Data management Intrusion detection system Whiteboard Googol Normed vector space Gamma function Website Library (computing) Address space
Presentation of a group Server (computing) Service (economics) Natural language Multiplication sign Execution unit 1 (number) Maxima and minima Set (mathematics) Mereology Neuroinformatik Revision control Data management Personal digital assistant Energy level Traffic reporting Library (computing) Link (knot theory) Surface Software developer Computer file Analytic set Client (computing) Cloud computing Volume (thermodynamics) Open set Coding theory Googol Sample (statistics) Medical imaging Thumbnail Quicksort Traffic reporting
Source code Information management Metric system Link (knot theory) Table (information) Code Natural language Authentication Dimensional analysis Analytic set Ext functor Google Analytics Client (computing) Electronic mailing list Infinity Formal language Googol Sample (statistics) Query language Googol Traffic reporting
Dataflow Digital filter Service (economics) Table (information) Computer file Code Natural language Authentication Google Analytics Online help Client (computing) Authorization Library (computing) Source code Link (knot theory) Information Sampling (statistics) Dimensional analysis Client (computing) Type theory Googol Process (computing) Sample (statistics) String (computer science) Website Right angle Communications protocol Library (computing) Traffic reporting
Source code Digital filter Electronic data interchange Service (economics) Key (cryptography) Information Code Authentication Dimensional analysis Client (computing) Client (computing) Sign (mathematics) Sign (mathematics) String (computer science) Authorization Right angle Library (computing)
Filter <Stochastik> Digital filter Mobile app Metric system Natural language Google Analytics Client (computing) Dimensional analysis Hausdorff dimension Source code Drum memory Summierbarkeit Library (computing) Game theory Link (knot theory) Demon Dimensional analysis Analytic set Core dump Client (computing) Menu (computing) Bit Googol Sample (statistics) Query language Right angle Metric system Library (computing) Traffic reporting
Authentication Point (geometry) Scripting language Information Computer file Dependent and independent variables Code Connectivity (graph theory) Analytic set Mereology Metadata 19 (number) Graphical user interface Emulator Semiconductor memory Query language Function (mathematics) String (computer science) Source code
Scripting language Raw image format Email Computer file Interior (topology) Skewness Analytic set Coma Berenices Bit Digital object identifier Cartesian coordinate system Mereology Digital object identifier Mathematics Spreadsheet Googol Personal digital assistant Query language Source code Quicksort Game theory Spacetime Row (database) Vacuum
Identifiability Online help View (database) Direction (geometry) Similarity (geometry) Bit Peer-to-peer Uniform resource locator Word Type theory Intrusion detection system Electronic meeting system Website Right angle Information Quicksort Gamma function Sinc function Physical system
Email Game controller Social software Mereology Cyclic redundancy check Revision control Intrusion detection system Representation (politics) Gamma function Drum memory Summierbarkeit Maß <Mathematik> Physical system User interface Source code Execution unit Pay television Link (knot theory) Information Moment (mathematics) State of matter Bit Inflection point Sign (mathematics) Website Self-organization Freeware Hydraulic jump Row (database)
Email Execution unit Email Electronic data interchange Identifiability Touchscreen Social software Link (knot theory) Software developer Lemma (mathematics) MIDI Password Emulation Sign (mathematics) Physical law Website Landau theory Gamma function Amenable group Address space
Trail Presentation of a group Token ring Computer-generated imagery Demo (music) Set (mathematics) Web browser Client (computing) Area Number Telephone number mapping String (computer science) Intrusion detection system Query language Information Video Genie Source code Programming language Touchscreen Information Video tracking Token ring Analytic set Client (computing) Ext functor Term (mathematics) Cartesian coordinate system Sign (mathematics) Type theory Radical (chemistry) Uniform resource locator Integrated development environment Query language Personal digital assistant Password Website Quicksort Sinc function
Source code Execution unit Computer file Twin prime Token ring Sheaf (mathematics) Electronic mailing list Bit error rate Analytic set Digital object identifier Line (geometry) Mereology Uniform resource locator Content (media) Personal digital assistant Query language Different (Kate Ryan album) String (computer science) Query language Statement (computer science) Authorization Row (database) Data type
Revision control Spreadsheet Presentation of a group Personal digital assistant Cellular automaton Lemma (mathematics) Cellular automaton Lace Quicksort Library (computing) Open set Library (computing)
Boolean algebra Rounding Computer file Information Code Cellular automaton Cellular automaton MIDI Grand Unified Theory Mereology Open set Wiki String (computer science) String (computer science) Order (biology) Source code Right angle Summierbarkeit Hydraulic jump
Scripting language Addition MUD File format Planning Bit Bit rate Drop (liquid) Computer font Graph coloring Formal language Spreadsheet Process (computing) Googol Emulator Googol Source code Quicksort Traffic reporting
Group action Service (economics) Computer file Token ring Code Multiplication sign Mereology Expected value Programmschleife Bit rate Different (Kate Ryan album) Googol Query language Error message Scripting language Validity (statistics) Structural load Lemma (mathematics) Token ring Analytic set Bit Group action Cartesian coordinate system Limit (category theory) Process (computing) Googol Error message Query language Quicksort
Point (geometry) Scripting language Source code Presentation of a group Inheritance (object-oriented programming) Demo (music) View (database) Code Demo (music) Electronic mailing list Menu (computing) Formal language Uniform resource locator Spreadsheet Googol Googol Free variables and bound variables Cuboid Convex hull Website Resultant
Mobile app Arithmetic mean Electronic data interchange Java applet Different (Kate Ryan album) Code Website Menu (computing) Representational state transfer Formal language Library (computing)
Source code Mashup <Internet> Functional (mathematics) Manufacturing execution system Link (knot theory) Authorization Electronic mailing list Sheaf (mathematics) Physical system
Multitier architecture Presentation of a group Mobile app Identifiability Social software Code Token ring Multiplication sign MIDI Password Function (mathematics) Thomas Kuhn Bit rate Term (mathematics) Intrusion detection system Physical law Summierbarkeit Error message Scripting language Source code Information Weight Lemma (mathematics) Port scanner Limit (category theory) Sign (mathematics) Number Web 2.0 Googol Self-organization Key (cryptography) Quicksort Fingerprint Row (database)
Google Analytics
welcome everyone to the webinar today the topic of the talk is about the api's from google and orchid and how to combine them the speakers for this talk is a this chris knowledge from orchid liz is part of the orchid technical team and she is one of the people who is working on the interface design and development of orchid in addition Liz's also involved with automation of work with services and training technologies in the orchid ecosystem I met Liz initially in open repository conference when she was giving a talk about orchid API at that time we thought okay well this is it this can make a very good webinar and that's why they are here so with that introduction I Liz I'm handing this to you okay great thanks so much of your I am happy to be here today's do
this ance webinar and as I mentioned today we're talking all about API so particularly about I google api's and orchid api's but there are some topics that will sort of apply generally and so as of yore mention i am a developer at orchid which means that that my work kind of revolves around api so the technical team at orchid most of the work that we do I is involved in building api's but I also do a lot of work with internal applications that or get to things that support our our internal development and and operations so a lot of what I do is about getting all of our different systems to talk to each other so that we can automate some of our tasks than make everybody's life more efficient and more convenient and that involves using a lot of API so I do work of building api's and on using them and through that experience I think I have a few tips and tricks that I can hopefully share with you today so we
could start out with a little with a little background so I know everybody has sort of different perspectives and maybe different technical level so sort of grounded in some basic eyes some basic discussion about api's um so we've got all sorts of things up in the up in the cloud these days so we've got things like photos and documents data and applications running in the cloud that we can access at any time from from any computer this is fabulous so we can we can get that data and run those applications and and do all sorts of great different things but sometimes
there is not just one single data source or one application that does everything that we want in order to do something very specific you might need data or features from multiple different sources and applications so that's where api's come in so there are loads of api's out
there for lots of different popular application facebook dropbox amazon slack google drive all sorts of applications that we can combine two different ways to how to do lots of different things so that's what we'll be focused on today's to the idea of combining these applications to get
exactly the functionality that we want but first we'll start out with what is an API so we hear that term tossed around a lot so API stands for application programming interface what it really means is a set of rules that allows computer applications to talk to each other um and that those sets of rules aren't really universal there's no sort of governing body that that defines exactly how an API has to work really software developers to develop applications get to decide how the API for their application gets to work there are some common sorts of sorts of architectures that lots of software developers you use that they all behave a little bit differently and have sort of subtle nuances to them little quirks that make using multiple api's together kind of a challenge so that said why would you use an API so in those cases api's allow you to do the same sorts of things that you can do in an applications user interface but just a little bit more so for example I in the Google Drive API you can do things like create a new document or in a Google Drive face you can create a new document you can do the same thing in the API but you have a little bit more control over the behavior also you can do things faster and you can automate repetitive tasks so if you need to create 900 Google Drive documents you certainly don't want to sit there and do that by by hand in the user interface you can use the API to automate that a little bit and finally what we're focused on here today is the fact that when you start using api's you can combine data and functionality from all sorts of different applications into your own custom application and maybe even combine it with some functionality and some data that you have in your own organizations application to come up with exactly the right thing that you need so in this session as I mentioned
we'll talk about a couple of different not api's that I've been working with quite a bit to automate some of the internal reporting that we do at orchid so we'll be talking about the google analytics API the Google Drive API Google sheets as well as The Orchid AP is and the sort of project that this is built around is creating a a custom Analytics report that's uploaded to Google Drive as a spreadsheet and then combined with some data that we pull in from the organ API so before we dive
into those in case anybody is not too familiar with these tools first of all Google Analytics um is a really popular free to a certain extent a tool that allows you to track user behavior on a website that you own so things like how many users are visiting your site what pages are they looking at what country are they coming from and you can customize that even more to get more granular information about what people are are doing on your website next up Google Drive so that's a pretty popular common cloud file storage and sharing application from google and living within that is the google the sheets application which is basically excel living in Google Drive and finally we have orchid so if anybody is not familiar with with orchid so it or could we run a system of persistent identifiers for academic researchers and we also allow them to create a digital record of their scholarly contributions so things like publications that they've that they've authored affiliations more recently peer review service that they've done that all sorts of other aspects of their scholarly record so working with those that handful of different AP is that the steps that will be going over in this session is our querying the analytics API I'm setting up Google API credentials um getting analytics data from the API uploading it to Google Drive as a spreadsheet then setting up or get API credentials and getting some data from the orchid API and adding it to drive so I know this is this is kind of a specific application that seems to focus around analytics and an orchid but really once we get into the to the Google API and setting up setting up credentials particularly those aspects are applicable to almost all of them the Google API so there's kind of something in there for for a lot of different use cases and I should
mention I did close the the link in the chat box but I have the materials for this entire session including the the
slides and some code samples and a whole
lot of of instruction on on setting up
and using the the code samples in in
github so if you want to check that out
follow alone or just look at that later it's all right there in github for you alrighty so we're going to start out
looking at the google analytics API and using the analytics API is sort of sort of assumes that that you have a few prerequisites the first one being since the analytics API is all about tracking user behavior on website right the first prerequisite is that you have a website so for the sake of this demonstration I have a little sample
website mine is a sample institutional repository that in theory allows users
to to download publications from the site and for some reason my download buttons have recently disappeared that's
the site there's a link to it in the in and github so we have a site next prerequisite for working with Google Analytics API is that you have set up google analytics account and configured your website within that account so this part I'm not going to walk through step-by-step because we're more so concerned with with getting into the api's but the link and some resources for getting that set up our are in the slides and in the in the github documents so finally um so you can get some basic information about who's visiting your your site and what they're looking at with just a basic Google Analytics setup without doing any further customization but depending on on your site you might want to set up some custom custom tracking so in my case I for my repository site I want to know which items users are downloaded so I have in advance set up some tracking on the on the download links for my for my publications on my website and that tracking I set up through a tool called the Google tag manager which is kind of a new Google tool that allows you to set up customized tracking you know in a user interface rather than just straight encodes it makes it a little bit a little bit easier so assuming we have those things in place we can then dive into the the analytics API and in in
sort of a perpetual parade of handy tools that that Google Offers um when you start querying the API it's really handy to use to use this tool the API the analytics query Explorer so what this does is allow you to sort of build queries visually and and see what the data that you get back from these queries is without having to commit them to code and run it so it just makes it simple to sort of play around with the API so once you open up the query Explorer you'll be prompted to to sign into your Google account and it will automatically pull in your any Google Analytics sites or any sites that are set up with google analytics into this section so you don't even have to know anything about your account or or the website property ID you can just get right into the queries so looking at the analytics queries so what we're looking at here is basically the API version of what you might see in the in the analytics dashboard so the dashboard
looks like this it's kind of a pretty pretty user interface to the analytics
analytics data but you're kind of limited to what Google provides you with here and it leaves within the analytics API dashboard so you can download reports but you can't really mix up the data with other with other data to create anything anything custom so if we flip over to the query Explorer we can
pull out some of the same data that we see in that user interface but just he and I in text we can really customize the queries to get exactly what we want so some of the aspects that you can customize here of course are the start and end dates for your for your report and then we get into a couple of other pieces metrics and dimensions so in
Google Analytics there are many many dimensions and metrics that you can that you can get data on and what a dimension is is how you want to break down the analytics data so do you want so they like users by the city that they're located in by the device that they're accessing so it's kind of the buckets of of data that you want metrics on the other hand are the things that you're counting so whether it's clicks or page views or something else entirely it's what you're counting so so you're putting the metrics into your dimensions buckets um those are the kind of the two key components of the of the analytics API and there is a huge reference to UM
to those called the dimensions and metrics Explorer that's look that lets you look through all of the different
dimensions and metrics and see what they
mean that's sort of helpful to have to
flip back and forth between the Explorer and the dimensions the metrics Explorer that query Explorer to get what you exactly what you want finally the other really handy field that's available is filters so we don't necessarily want all of them all of the information from the dimensions and metrics we might want to know something about users who only visited our homepage or something something like that so we can use filters to I get just the data that we just the subset of data that we want so from my sight I've got this query sort of set up my metric is total events to total sort of user interactions with my site and then the dimension that I want is event label and this is specific to some of the custom tracking that I've set up so some of my some of my events events being clicks and downloads have have tracking attached to them that will tell me what the DOI of the item that was clicked or downloaded is and filters in this case I only want to know I only want information about things that users have downloaded so there are plenty of other fields that you can set here but I'm going to focus on dimensions metrics and filters because those are kind of the bigger the big key pieces I can run my query right in this window and what I get back
is not only the query results but a link to the report that I can go back to and most importantly for for developers you get the full query URI so when you go to to code something something up using this query you have all the bits and pieces that you need are right there that you can drop into your oh alright
so we've got our analytics query query ready to go now we need to get access to the Google API is just starts to start writing up something that can take that query and run it against them the api's alright so so next up we need to get access to the Google API and this is where for me getting started things started to get a little bit tricky because they're just a lot of steps two steps to go through that are not completely obvious at all stages so to access Google api's you need a set of credentials so something sort of like a username and password for the for the API since Google and most other API providers depending on the provider don't necessarily let anyone in off the street interact with their API they have to know something about who you are what you're doing before you can get into the to the API so for Google abis to get
access to any of the VAP is you need to create Google developer account and a Google developer project so we'll slip
over to Cooper's so this is just another sort of Google account that you can connect to you can log right in with your existing Google account and what you'll need to do first is create a project so when you first set up google developers you'll be prompted to create a project right away otherwise if you have an existing account you can go up to the top menu and click on create project the URL by the way up here is consoled app developer google com you can also get there through just developers google com really just sort of a bucket to put your API credentials into it doesn't necessarily have to be related to a specific coding project it is useful for keeping credentials separate and sharing them sharing them with different groups of users for example at orchid we pretty much use 11 google developers project to store all of our all of our Google Google Apps credentials so I'm gonna go ahead and create a new project and this tense sometimes takes a few seconds
depending on fast the Google API sore
running so you see that still spinning up in the upper right-hand corner so
once we get set that set up the next step will be to I enable different Google api's for that project there are hundreds of Google API s and they don't all come pre enabled when you set up a new project you do have to add them one by one so once your project is set up
you'll automatically be shipped over to the section where you can add api's the
first one that we want to add is the analytics API it's great we'll click
enable and so just as google says this
api is enabled but we don't actually have credentials for it so we can use it yet so the next step is to click this
button and add some credentials so at
this point when we're talking about credentials this is this is again sort of that that user name and password concept to access the API and google has a few different types of credentials that you can I get for different api's depending on the type of project that you're working on and this is where things can get a little bit tricky if you are a developer you might be familiar with with OAuth authorization protocols and that's what google uses for its api's I'm and there are a few different a few different types of co-op authorization protocols some of them require a user interaction and some of them don't so it depends if you have an application that you want to be able to run all by itself on a server or well any machine without without somebody having to type in a username and password or or do something that involves a browser or at a user interface every time you want to run the application and then you need to set up access through what's called a service account so these other these other options particularly the API key require some sort of user interaction so we're going to focus on the service account authorization today because it's kind of the trickiest and also really useful because we can set up an application or a script that we can set to maybe run automatically every so often without anybody having to touch it or interact with it and so this little widget is designed to sort of tell you what kind of credentials you need but I and I know that we already need a service account so I'm going to click service account
and what a service account is is kind of a shadow user in Google that can do things with the with the API but it's not really a person it doesn't have all of the the same privileges that a person does but it can access different Google API resources on on behalf of an application so I'm going to create a new service account for the sake of this
demo I'm going to make it a project owner so it can have all sorts of access to do all sorts of different things permissions that you want may vary depending on your project so here's our service account ID that that will need for gym later we also want a new private key so this is a file that that will download and we'll use that sort of in lieu of a password so this will be basically our password for the for the API and I'm going to make this p12 format depending on what you're what you're doing especially if you're working with with Google Apps for Business where you have your own domain you might also want to enable Google Apps domain wide this allows your service account user gives your service account user access to to everything on the on the Google Apps domain on the API side it doesn't automatically give them sharing permissions for those things but it allows them to it allows the service account to access those things through the API if sharing permission is granted so I'm going to enable that and for the
sake related to some of the other Roth authorization flows you do need to enter a product name even though it might not be useful for your particular use case so we'll create that and it's I
prompting us to download this p12 key file which is that thing that will act as our service our password in our code later alright so that's all that's all
set to go as we've got analytics enabled the other the other app that we're going to need access to is google drive and we can go ahead and activate that at the same time so this process is pretty much the same for all of the different Google API so I'm gonna go back to the library
and pick out drive API enable that and
now since we have that service account
already configured we don't need to go
back and configure credentials again for that it's just enabled and and it's set to go well the other one that we need is
the sheets so that's set to go as well
so when we're working with the the analytics API the one last thing that you also need to do is add your service account user on to your analytics
account so that's one one step that could kind of trip you up and so that's under user management in your in your analytics administration so you just add that person like any other user using the the kind of fake email address the user ID that was generated when you set up your service account finally to get
ready for using using google drive will
need to create a folder and give our
surface out access to it so I've actually already done that in in dry but the
reason being so we want to create sort of a custom report from analytics and drop it into a Google Drive folder it's a little easier if we already have that drive folder setup and also also shared
with that that service account all right so all the parts and pieces are set to
go that's a pretty tedious set of steps and I have them listed all in the island github documentation but now it's sort of sort of time to time to get into some coding or at least talk about what we can do with all of that set up so before we go on are there any questions about configuring access to Google API nice well Liz I have one question we we talked about creating the google account for a google developer account for google services so i know some of the Google API czar are not free like the Google cloud services for example you to pay for it if you use beyond certain units units of computation so if for API is that you are using in this presentation would it be correct to say all of them are free API all the ones we're using in this presentation are free api's yes um certainly when you there are when when you get into higher volume usage of the analytics API there is a paid version of the of the analytics API or an upgrade but the ones that we are that were talking about today at a basic level are free all
right so we're all set up and we can finally get back to writing those queries and getting analytics data and uploading it into into drive so the code examples that that I'm going to use use Python but the but all of these things
certainly can be done in in any
different any language that you prefer to use the one thing to keep in mind is that Google has has libraries for for quite a few languages so this is the google analytics tation um all of the other api's have
documentation that's formatted similarly and most of them have have pretty robust client libraries that that make interacting with the API is really really easy so that's the first thing that you're going to want to I get started with and if you do look at the code samples you can see that we import
the two clients and the Google API client right off the bat so with those
with those client libraries uh now available and up and running the next step is to authenticate to the api's that you that you want to use so this is where that that p12 client secret file comes in um when you're using a service account at least and the type of oooff to authorization so for those of you who are too may have heard of go up before familiar with with authorization protocols the type that we're using that's kind of the important keyword to look up if you're if you're searching stack overflow or or just googling for our help assigned at JWT assertion credentials it's kind of a less common lots to authorize ation flow and there's not a lot of information on the Google the google web sites about it there's kind of an overview document that describes the whole process but that was that was definitely a tricky piece to UM to figure out so i definitely recommend either taking a look at at this code
sample or or doing some searching for
sign JWT assertion credential authorization if you have any trouble i get any authorization piece done but in
on the python side right over here this is where we're doing the the authorization so using using the Google API client libraries we are sending in our some information about our service account or key and the API that we're looking to to authorize into so that piece works the same for all of the api's that that we're working with so it's a piece of code that you can you can reuse across an application so next up getting
back to those queries that we looked at
in the query Explorer over here um we
can now take that those bits of that query and put it into in two coats if we look at the URL app we've got our our metrics dimensions and filters that we need to send into the Google API all all right down here and right here is where
that where the were those pieces get
sent into there's it within the google client library for for the api the analytics API we've got some got some methods we can just send that data right into and run those on those queries so
finally I'm uploading data to Google Drive so we do the authentication the same the same ways as analytics but
instead of sending in our query information we send in some some file metadata and back it back over in the in the query here when we got that on when we got that query data back I put it into basically into a CSV file in memory and then we're passing that over as the is the data for the Google Drive file so in in the part of my part of my code that actually controls all of the the scripts at this point if I comment out the the orchid API components and and
run the script should get back a file
that has some fair analytics data in it there we go so it's got though did I
just ran it up so this is what we have
so it's not a terribly fancy file but but we've got our data out of out of analytics and we could see some of the same bits and pieces that we saw in the query Explorer also I have some some headers that I added in the in the script itself so we've got all that into into a CSV and uploaded into google drive and the nice part about this is once you've got it in Google Drive you could share it with all with any users you want I'm and also allow other users or other applications to make changes to it and that's what we're going to do next so I have some sort of blank spaces in here so I can see how many how many users have downloaded some of my do is but in this case I want to know how many of those digital object identifier are connected orchid records so got some space in here to add that that information and next up we're going to query the orchid api's and then use the sheets API to to edit this existing spreadsheet so we're going
to switch directions a little bit and talk about the orchid API so again if you're not too familiar with orchid I'll
flip over to the orchid orchid website so again this is a digital identifier
for researchers that can be linked to publications and I and funding and peer review activities and a few other types of activities to to help attribute those scholarly contributions to the right person since lots of people have the same or similar names it's really
helpful to have a unique digital identifier linked to those individuals instead of just trying to connect those items to two names or to sort of internal identifiers that don't don't cross communicate well with other other systems so the whole orchid identifier
also also includes the concept of an orchid record so we have this visual
representation of our researchers
activities here in the orchid record there's a public version as well so this
is what you would see in the in the website user interface but you can also
get at all of this publicly visible data through our our AP is so we offer two
different api's one is public it's freely available that's the one we're going to be taking a look at today so that allows you to search and retrieve all of the information that's publicly visible in the in the orchid so the things that you can see on the website you can also get get through the through the api's and another feature is that it allows you to get authenticated orchid IDs from users which is not a part we're going to be talking about at the moment but it is there we also have a paid member API that also allows member organizations to ought to write data into the system and also includes a few other other features um but we're focused on free api's today so we'll talk about getting access to the public API um so like the Google API sui also require users to to generate to use credentials when they access the API so we have a little bit of information a little bit of control over how the API is being used and so to do that you
first need to create an orchid account also create an orchid identifier and it
doesn't matter if you're not not
actually a researcher it's perfectly fine to create an orchid account so to
do that you can just go to the orchid
website and click any of you of the amenity register for an orchid ID links
that we that we have all that you need
to ought to give us is a name and an
email address and then you'll be set up with your with your orchid account once you sign into your account you'll see at the top of the screen that you have this
developer tools tab and that's the spot
where you can go to create some API credentials flip back to the presentation since I've already created
some of that account so if you haven't created credentials yet you'll see a big blue button that says register for the orchid public API once you click that you'll see a screen that prompts you to type in some information about your public API application and that doesn't necessarily need to be too specific for the for just searching the API and once you say that you'll get a set of credentials that consists of a client ID and client so kind of like a username and password so those are the two pieces that you'll need to to get information from the AP is so once you have those things you can use those to query and to get authenticated IDs and in the next steps we are going to use those credentials to ask the orchid API about which orchid IDs are connected to the do is that we're tracking in analytics so to get that information there are kind of their two steps in the API and I'm going to show those using I just some basic HTTP requests um which can be can be run in basically any any programming language so for those of you who aren't quite sure what that means so HTTP requests that's sort of the same as what you do when you when you visit a website in a browser only when you're using a sort of a terminal application like curl I you can do quite a few more things with HTTP requests and you can do with just a browser URL bar alright so first step we need to use that client ID and client secret to get an access token so the access token is what really allows you to to run your queries so you need to send that token which is a big long string of letters and numbers that you'll send along with the with the query and in this case we need to send in some information about the scope so what we're looking to do with the api's as well as the environment that were working with so once i get that token we can use it to run some queries on the orchid api and we do have quite a bit of
documentation on searching so lots of different parts and pieces of users or kid records that you can search so we have keyword searching and fielded searching as well so for this case we're focused on searching for digital object identifiers and that query looks looks
like this here on that line when we translate it over into
into Python and this is this orchid API dy file the query section looks like this so we're just sending a curl statement with that same query string and in this case we are taking that list of DIYs from analytics and just repeating that query over and over again for those do is and finally as i
mentioned we're going to edit the existing spreadsheet to to add the orchid data in so when you're using the sheets API so when I put together this presentation the the newest version of the sheets API hadn't yet been released and now it now it has and so previously it was pretty common especially in Python to use another library on top of the sheets API because the sheets API is is a little finicky doesn't have a lot of the features that that are sort of needed to make it really useful and so I'm using the Python geez breath library which just makes it easier to do things like search for data in an existing spreadsheet and update update cells in a spreadsheet in a batch on since the newer version of the of the Google sheets API has been released there are quite a few more features it still doesn't have the have searching built in Sochi spread is still still useful in certain cases so what we're doing in the
eye with the sheets API portion of this
code is getting a drive file ID of that
of that file that that we created so
every Google Docs file you'll see at the
end of the in the URL string has a file ID so that's the important part that you need in order to be able to manipulate the file in fact folders have ideas as
well and you use those in the and the API to find the find the right folder and list of files inside and then in
this code all that we're doing is looking for for the cells that contain the do is that we're looking for and dropping those pieces of information from the orchid from the orchid API in X to them
so if I uncomment the orchid bits in my
in my report script and run it again we
should see those things drop in and this will take a few more minutes because um the sheets API is kind of slow and and
also we're querying orchid as as well so we're adding a few more steps to the process so well that's running we can talk about just a few more things that you can do once you have that data in into something like a Google spreadsheet and so we've got it in addition to just the the plane sheets API you can add in charts from the Google charts API and google also has sort of a javascript like language that lets you do some some formatting so you can add fonts and colors and make your sheets look a bit prettier of course you could also pull
that the data from that sheet out using the drive API and um manipulated with
some other application a couple of tips when you're using the google api's that kind of tricked me a few times were that um so when you're using those p12 files you are actually and authenticating to the Google API so you're actually generating a token in the process that's just passed around in your application unlike the that work at API tokens which are valid long term so they're valid for 20 years Google API tokens expire in an hour um so if you're running code that loops through a lot of different files or folders or whatever um you just need to make sure that you that you account for the fact that you need to have a valid token and your tokens will expire in an hour secondly working with free API so it's great that they're free but it also comes with comes with the expectation that you might not always have perfect access to the api's all the time so Google API is particularly since they're popular they can have a heavy load that load can vary they're a little bit twitchy um you'll get some sporadic errors just sort of unexplained 500 errors the service isn't available sorts of things and that's just part of part of working with free ap is especially when you're running lots of queries over and over again and particularly with the analytics API on their def query limits and rate limits that you can run up against if you're running lots of queries over and over again finally as I mentioned the streets API is pretty slow so it's good to try to combine as many actions into one request as possible I think my script should be finished now and it is all
right and I can see that my orchid API data was dropped in to this couple of spots where i had placeholders for it alright so again um the code for this is
all all available in github as well as the presentation and a whole lot of URLs for the resources and think we can go ahead and maybe hates that URL into the chat box again but at this point I think
we have a few minutes left for for questions and so I say there are a couple tonight in the chat box so one is you could choose to store your resulting spreadsheet locally certainly that as yeah that's definitely a possibility um and not in Google Drive yep definitely definitely possible um but for the sake of this demo one of the one of the Handy things about living it to drive is that you can share it with other people either at your organization or or outside of it sorry list for the purpose of recording I just actually repeat that question which the question was can you actually store the spreadsheet or a result of the script into the local drive rather than the Google Drive if you already answer now the next question here is that so the a million actual question is what language is that does the orchid API support I'm sorry api is
a is a restful api so it's you interact with it using HTTP requests and so it with that in mind you can interact with it with any language that um that can send and send and receive HTTP request so we have some libraries out there for for everything ranging from PHP to JavaScript Java also Python and we have a couple of people that are using that as well but anything that you can think of that can can make HTTP requests we do on our so our support site is our documentation site is
members orchid dot org and it doesn't mean that you actually need to be a member to to access it but there you'll find all of the documentation for a PS and we also have some example example code and some libraries for different for different languages we even have some I have a go app Ruby definitely
okay well that is a comprehensive list well we have one more question somewhat related to the API so the question is I'm just really in the actual Christian and then I actually add something top of that so the question is whether the orchid API can be used to link publication funding grants info into the authors perhaps using the crossref API so if I understood the question here correctly is actually talking about a
mash-up of IEP is using orchid to actually find what publication or grants can be linked to that so is there any function like this in orchid system um so that would be to find in which which working identifier czar linked to to funding so we do have a funding section on on the orchid record that either um
either users themselves can add funding items to or other member or other
organizations outside of orchid can actually populate funding information
into users orchid records so you can you
can get funding information out of the
out of the orchid API and again just check the documentation you can you can see where to get those things there also um so there are some some funding agencies that are that are putting putting orchid identifiers into you into their data crossref fun ruff some items have I have orchid identifiers in them we have so over is the is one of the organizations that we work with that also includes orchid orchid IDs or allows allows agencies to populate orchid IDs and that's got a lot of federal funding information that's not to say that all of those grants have work identifiers populated into them but we are seeing more and more organizations that are adding orchid identifiers to their to their funding items and then populating them back into to orchid okay we I think let's just answer one more question because we are technically your time limit and so the last question is that what sort of ways or methods do you can use basically to cope with the API errors so that's a that's a really good question particularly with with Google API since the errors can be really unpredictable and not you know at all related to onto your code so I some apps are pretty long running scripts in I ate that use google api is to be honest have some pretty blunt i work around so one thing for the for the token check is i have in some of my scripts a sort of a time-based a token check that starts a new that checks the the current time when the token when i get the token and then it just checks routinely throughout the script to see if it's below the token expiration time and if it's above a certain time it just gets a new chuckling automatically certainly you could also check to see if the token is valid but because of weight limits and and api errors i like to keep that check on the on the script side so i'm not using up my quota in terms of handling just the random errors I if I can't can't sort of monitor the logging output as it's running I'll sometimes just use a bash script to to retry it until it succeeds that can present some errors if you're if it's retrying because your your code is because there's something wrong with your code you'll hit that api rate limit really really quickly as your code rejoice over and over again but those are some of the things that that I've been doing okay thank you very much miss that was a great presentation thanks everyone for coming to this talk
and I hope to see you in the next and next lebanon