Using Google's APIs with Python scripts

I was never able to find centralized, succinct and example based documentation for doing domain delegated API calls with Google. Hopefully here is exactly this documentation from all the pieces I gathered along the way.

Service Account Creation

  1. Go to https://console.developers.google.com/start and create a new project.
    Screen Shot 2016-03-15 at 10.11.31 AM
  2. Call it whatever you want
    Screen Shot 2016-03-15 at 10.11.48 AM
  3. Enable the right APIs that this project will use We’ll do drive API for the purpose of this testing
    Screen Shot 2016-03-15 at 10.15.06 AMScreen Shot 2016-03-15 at 10.16.40 AMScreen Shot 2016-03-15 at 10.16.48 AM
  4. Go to the “Credentials” screen
    Screen Shot 2016-03-15 at 10.18.08 AM
  5. Create a “Service Account Key”
    Screen Shot 2016-03-15 at 10.18.33 AM
  6. Make it a “New service account” and give it a nameScreen Shot 2016-03-15 at 10.19.17 AMScreen Shot 2016-03-15 at 10.20.09 AMScreen Shot 2016-03-15 at 10.23.45 AM
  7. Download that JSON file that follows.
    Screen Shot 2016-03-15 at 10.23.53 AM
    This file contains the credentials for the account you just created, treat it with care, anyone getting their hands on it can authenticate with the account. This is especially critical since we are about to grant domain delegation to the account we created. Any one with access to this file is essentially able to run any API call masquerading as anyone in your Google Apps domain. This is for all intents and purposes a root account.

Domain Delegation

  1. Back on the “Credentials” screen, click “Manage service accounts”
    Screen Shot 2016-03-15 at 10.26.43 AM
  2. Edit the service account you just created
    Screen Shot 2016-03-15 at 10.28.23 AM
  3. Check the “Enable Google Apps Domain-wide Delegation” checkbox and click “Save”.
    Screen Shot 2016-03-15 at 10.30.28 AM
    Google at this points needs a product name for the consent screen, so be it.
  4. At this point, if everything went well, when you go back to the “Credentials” screen, you will notice that Google create an “OAuth 2.0 client ID” that is paired with the service account you created.

Domain delegation continued, configuring API client access

Granting domain delegation to the service account as we just did isn’t enough, we now need to specify the scopes for which the account can request delegated access.

  1. Go to your Google Apps domain’s Admin console.
  2. Select the Security tabScreen Shot 2016-03-09 at 11.15.40 AM
  3. Click “Show more” -> “Advanced Settings” Screen Shot 2016-03-09 at 11.15.52 AM
  4. Click “Manage API client access Screen Shot 2016-03-09 at 11.16.08 AM
  5. In the “Client Name” field, use the “client_id” field from the json file you downloaded earlier. You can get it via the following command:
    cat ~/Downloads/*.json | grep client_id | cut -d '"' -f4

    In the “One or More API Scopes” field use the following scope:

    https://www.googleapis.com/auth/drive

    Screen Shot 2016-03-15 at 11.00.36 AM
    If you want to allow more scopes], comma separate them. This interface is very finicky, only enter URLs and don’t copy/paste the description that show up for previous entries. There also might be a few minutes delay between you granting a scope and its taking effect.

  6. Click “Authorize”, you should get a new entry that looks like this:
    Screen Shot 2016-03-15 at 11.01.51 AM
    If you need to find the URL for a scope, this link is helpful.

Scripting & OAuth 2.0 authentication

Okay! The account is all set up on the Google side of things, let’s write a Python script to use it. Here’s your starting point:

google_api_script.py

This scripts contains all the functions to get you started with making API calls to Google with Python. It isn’t the simplest form it could be presented in but it solves a few issues right off the bat:

  • All Google interactions are in the “google_api” class, this allows for efficient use of tokens. When “subing as” a user in your domain, the class will keep track of access tokens for users and only re-generate them when they expire.
  • Exponential back-off is baked-in and generalized to anything unusual gotten back from Google (based on observation).
  • SIGINT will get handled properly

Before running the script, you may need to:

sudo apt-get update && sudo apt-get install python-pycurl

Running the script is done as such:

./google_api_script.py /path/to/json/file/you/downloaded/earlier.json account.to.subas@your.apps.domain

It will simply run the “get about” Drive API call and print the result. This should allow you to verify that the call was indeed executed as the account you specified in the arguments.

Once you’ve ran this script once, the sky is the limit, all the Drive API calls can be added to it based on the get_about function.

Important note on scopes: the same way that you granted domain delegation to certain comma separated scopes in the Google Apps Admin Console earlier; this script needs to reflect the scopes that are being accessed and the same space separated list of scopes need to be part of your jwt claim set (line 78 of the script). So if you need to make calls against more than just drive, make sure to update scopes in both locations or your calls won’t work.

More scopes & more functions

Taking it one step further with the Google Enforcer. This is the project that lead me down the path of writing my own class to handle Google API calls. While it is not quite ready for public use, I’m publishing the project here as it is an excellent reference to making all kinds of other Google API calls; some doing POSTs, PUTs, DELETEs, some implementing paging, et cetera.

Download:
google_drive_permission_enforcer_1.0.tar.gz

The purpose of this project is to enforce on the fly permissions on a directory tree. There is a extravagant amount of gotchas to figure out to do this. If you are interested in implementing it with your organization, please leave a comment and I can either help or get it ready for public use depending on interest.

This project works towards the same end as AODocs, making Google Drive’s permission not completely insane as they are by default.

Here are the scopes I have enabled for domain delegation for this project.

Screen Shot 2016-03-15 at 4.55.25 PMProblems addressed by this project:

  • domain account “subbing as” other users AKA masquerading
  • a myriad of Google Drive API calls focused on file permissions
  • watching for changes
  • crawling through directory hierarchy
  • threading of processes to quickly set the right permissions
  • disable re-sharing of files
  • access token refreshing and handling
  • exponential back-off