Google Drive connector

This tutorial will guide you through connecting Pathway to your data stored on Google Drive. For detailed information about Google Drive connector, refer to the API documentation.

Setting up Google Drive

To enable access to your Google Drive files through the Pathway app, you'll require a Google Cloud project and a service user.

Creating a new project in the Google API console:

• Go to https://console.cloud.google.com/projectcreate and create new project
• Enable Google Drive API by going to https://console.cloud.google.com/apis/library/drive.googleapis.com, make sure the newly created project is selected in the top left corner
• Configure consent screen:
  • Go to https://console.cloud.google.com/apis/credentials/consent
  • If using a private Gmail, select "External", and go next.
  • Fill required parameters: application name, user support, and developer email (your email is fine)
  • On the next screen click "Add or remove scopes" search for "drive.readonly" and select this scope
  • Save and click through other steps
• Create service user:
  • Go to https://console.cloud.google.com/apis/credentials
  • Click "+ Create credentials" and create a service account
  • Name the service user and click through the next steps
• Generate service user key:
  • Once more go to https://console.cloud.google.com/apis/credentials and click on your newly created user (under Service Accounts)
  • Note service user email address, it will be needed later
  • Go to "Keys", click "Add key" -> "Create new key" -> "JSON"

A JSON file will be saved to your computer. Move it to the folder where your Pathway script is located and rename it to credentials.json.

Sharing files with service user

You can now share desired Google Drive resources with the created user. To do so, visit your Google Drive page, right click on the resource and share it with the email associated with the service user you generated earlier.

Once you've done it, you'll require the ID of the shared file or directory. You can obtain this by right-clicking on the file, selecting "Share" and then copying the link. The ID will be part of the URL.

For files: https://drive.google.com/file/d/[OBJECT_ID]/view?usp=drive_link
For folders: https://drive.google.com/drive/folders/[OBJECT_ID]?usp=drive_link

Copy the OBJECT_ID from the URL.

Connecting to Google Drive with Pathway

Once you've finished setting up your Google Drive, you can now start reading from it using pw.io.gdrive.read (ensure to replace object_id with the one you copied in the previous step):

table = pw.io.gdrive.read(
    object_id="0BzDTMZY18pgfcGg4ZXFRTDFBX0j",
    service_user_credentials_file="credentials.json"
)

pw.run()

By default, the connector operates in streaming mode, observing all changes occurring in a particular file or folder. Refresh rate can be adjusted by passing the refresh_interval parameter. Static mode can be enabled by specifying the argument mode='static'. Further information regarding available parameters can be found in API docs under this link.