Step-by-step guide to using the Google Indexing API with Python

This guide serves as a manual for SEO professionals on using the Google Indexing API. The purpose of this guide is to utilize a Python script to bulk submit URLs to Google for crawling, aiming to resolve certain indexing issues with your pages related to indexed URLs that are crawled but not currently indexed. Follow the step-by-step instructions, illustrated with images, to utilize the Google Indexing API via Python.

Step 1: Installing Python

To work with Python scripts, you need to have Python installed on your computer (or laptop). If you already have it, you can skip this step. For those who do not, you will need to visit the official website at www.python.org, download, and install the latest version of Python.

I use a Mac, but if you use Windows, the steps will be identical.

Step 2: Configure the Indexing API Key in Google Developer Console

Navigate to the Google Developer Console. The Google Indexing API is free, so don’t worry about adding a billing account if you are only using the Indexing API.

Create a new project. Give it a name that is convenient and understandable; I will call the project “SEOBLOG”. Click the “Create” button.

Navigate to the newly created project (SEOBLOG project). After creating the new project, save it as the selected project as shown in the screenshot below, and proceed with the following steps.

Configuring API credentials and services. To do this, you need to set up API credentials by following the “Credentials” link in the “APIs & Services” menu on the left sidebar. Once in the “Credentials” window, click on “Manage Service Accounts” and create a new service account.

Create a service account. On the Service Account Creation screen, you can follow the steps displayed to configure your new service account.

Name the account with a memorable name that is related to your project. I am using the name “SEOBLOG Indexing Service Account.”

After you add the name and click the “Create and Continue” button, you will need to select a role that will grant your account access to the project.

Select the “Owner” role and click the “Done” button.

After you complete this step, you will be directed to the “Service Accounts for Project” page, as shown in the screenshot below. You will see the email address (gservicesaccount.com) associated with your project and service account.

Copy this email address, as you will use it in the next step in Step 3.

Create a key for your account. Click on the three dots under the action column and select the “Manage Keys” option.

You will be directed to the “Keys” page, where you can add a key by clicking on the “Add Key” button. Leave the option set to JSON and click the “Create” button. A small JSON file will be downloaded. This is an authentication file that will verify your access to the API. A pop-up will appear stating “Private key saved on your computer.”

Turning on the API. The next step is to enable the API. To do this, navigate to the “APIs & Services” section and select the “Library” option.

Find “indexing api” and select the first option, Web Search Indexing API, from the two options provided.

Click the “Enable” button on the next screen.

This concludes the steps of the section on setting up the Indexing API in the Google Developer Console.

Step 3: Granting the owner access to the GServicesAccount email in the Google Developer Console

At this step, you will grant owner-level access to the email address gservicesaccount.com, which was created in the previous step. If you did not save the email address in the previous step, you can view it by selecting the Credentials link on the left sidebar.

After you have copied the email address, go to your Google Search Console account:

1. Open your project in Google Search Console.
2. Go to “Settings.”
3. Enter the “Users and permissions” section.
4. Click the “Add User” button.
5. Add the service email.
6. Select the “Owner” status and click the “Add” button.

Step 4: Running the Python script

In the previous steps, we completed the setup of the API indexing key in the Google Developer Console, installed Python, and granted owner access to the API key in the Google Search Console. Now, we have moved on to the final step, where we will use the Google Indexing API with a Python script.

Installing libraries

Open the terminal on your Mac (or the command prompt if you’re using Windows). We will need to install some libraries before we can use the Google Indexing API. Run the following two commands in the terminal:

For Mac users:

Command 1: pip3 install oauth2client httplib2

Command 2: pip3 install pandas

For Windows users:

Command 1: pip install oauth2client httplib2

Command 2: pip install pandas

Necessary files

Create a folder in which to place the 3 files:

1. Python Script;
2. URLs.csv;
3. JSON file (which you uploaded earlier).

The JSON file uploaded in step 2 needs to be moved to this folder. Download the file urls.csv, which will contain the list of URLs you want to send for indexing to Google, and the indexing.py code from this link, and move them to the same folder as shown in the screenshot below.

Or independently create and add a copied Python script from github.com.

Let’s change the Python code and the CSV file

You only need to make a small change in line 8 of the Python code in indexing.py. Replace the JSON file name in this line JSON_KEY_FILE = “YOUR_JSON_FILE.json” with the name of your JSON file.

You can edit the file using programs such as Sublime Text, PyCharm, VS Code, or other IDEs.

Next, you will need to add URLs to column A of the urls.csv file. Do not delete the first row containing URL values and dates. You can add a maximum of 100 URLs at a time.

I mostly use Google Sheets, so I created a file and added the necessary 100 URLs to it, then downloaded it in CSV format (.csv).

Run the script

Open the terminal on Mac (or command prompt if you’re using Windows) and navigate to the folder containing three files. You can use the change directory (CD) command as shown in the screenshot below. Once in the desired folder, you can run the script using the command provided below.

For Mac users:

Command: python3 indexing.py

For Windows users:

Command: python indexing.py

A small step-by-step guide in the terminal

1. Open the terminal and type the command cd Desktop (if your folder is located on the desktop).
2. To verify that the folder with the files is indeed on the desktop, enter the command ls (in the example, we see that there is one folder named SEOBLOG on the desktop).
3. To navigate to the folder to run the script, you need to enter the command cd SEOBLOG (in your case, you need to enter the name of your folder).
4. To verify that the files we need are in the folder we navigated to, type ls (for example, the necessary files are located in the folder as shown in the file).
5. Enter the command python3 indexing.py.

After entering the command and pressing the input button, you will see a notification of updating the Google URL for all the URLs entered in your urls.csv file, sequentially, along with timestamps.

Google Indexing API limits for adding URLs

Daily limits:

• 200 URLs for requests to URL_NOTIFICATION_UPDATED and URL_NOTIFICATION_DELETED.
• 100 URLs for requests to URL_NOTIFICATION_MULTI.
• 180 requests for GET /v3/urlNotifications/metadata.
• 600 requests in total for all API methods.

It is also important that:

• The URL complies with Google Search Console rules.
• The URL should not be blocked by robots.txt.

Google may not index all URLs, even if they have been submitted via API.