Skip to content

After one hour get INFO:oauth2client.client:access_token is expired, but requests keep working #72

Closed

Description

@mozboz

I'm doing a large job to download approx 50k files from google drive.

After exactly an hour I start to see this message:

INFO:oauth2client.client:access_token is expired. Now: 2020-12-17 23:39:39.355448, token_expiry: 2020-12-17 23:06:32.235919
INFO:oauth2client.client:Updated access_token read from Storage

I have a single instance of GoogleDrive which I pass to all processes.

It seems like there must be a valid token as subsequent requests work fine, even though they continue to display this message.

What's the correct way to deal with this?

I'm using Python's multiprocessing to run many processes in parallel, and pass the GoogleDrive object to each worker process.

Activity

  1. shcheklein commented on Dec 28, 2020

    @shcheklein
    Member

    @mozboz could you please share a bit of your code please? How do authentication (once? every time you run it? from a file?).

  2. shcheklein commented on Dec 28, 2020

    @shcheklein
    Member

    @mozboz also, do you see it happening once an hour per thread or after an hour it keeps doing this per request, again and again? Does it print the same token_expiry: 2020-12-17 23:06:32.235919 date?

  3. melito00 commented on Jan 26, 2022

    @melito00

    I have same issue.
    authentication code as below. I can upload files after authentication, but one hour later access token was expired with same message "oauth2client.client: access_token is expired. Now: 2022-01-26 04:09:30.270165, token_expiry: 2022-01-26 04:09:29.818932'.
    Even though using Service Account Credential, oauth2client try to renew access token with client secret.

    scope = ['https://www.googleapis.com/auth/drive']
    credentials = ServiceAccountCredentials.from_json_keyfile_name(JSON_FILE, scope)
    drive = GoogleDrive(gauth)
    
    Traceback (most recent call last):
      File "/home/kyamada/work/crawler/lib64/python3.7/site-packages/oauth2client/clientsecrets.py", line 121, in _loadfile
        with open(filename, 'r') as fp:
    FileNotFoundError: [Errno 2] No such file or directory: 'client_secrets.json'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/kyamada/work/crawler/lib64/python3.7/site-packages/pydrive2/auth.py", line 442, in LoadClientConfigFile
        client_config_file
      File "/home/kyamada/work/crawler/lib64/python3.7/site-packages/oauth2client/clientsecrets.py", line 165, in loadfile
        return _loadfile(filename)
      File "/home/kyamada/work/crawler/lib64/python3.7/site-packages/oauth2client/clientsecrets.py", line 125, in _loadfile
        exc.strerror, exc.errno)
    oauth2client.clientsecrets.InvalidClientSecretsError: ('Error opening file', 'client_secrets.json', 'No such file or directory', 2)
    
  4. melito00 commented on Jan 26, 2022

    @melito00

    I only took a quick look at the code.
    If "auth_method" is "service" and the access token was expired, it looks ServiceAuth function is called.
    ServiceAuth function try to get value of "client_json_file_path" to get the credential info again, but "client_json_file_path" is not include SERVICE_CONFIG_LIST and I guess there is no way to set value of "client_json_file_path".

  5. shcheklein commented on Jul 24, 2022

    @shcheklein
    Member

    Closing as stale

  6. TheManchineel commented on Jul 28, 2022

    @TheManchineel

    This is still absolutely a problem. Just encountered it now. Any workarounds? The other login method for PyDrive2 (with client secrets et al.) is way too complex and overkill for what I need to do, and to my experience the credential renewals are super error-prone/requiring a sign in via browser every few days (courtesy of Google I guess).

  7. shcheklein commented on Jul 29, 2022

    @shcheklein
    Member

    @TheManchineel could you share the error one more time please? And your config, how do you setup it?

    does it happen after you run it for one hour+ or right after you launch it?

  8. shcheklein commented on Nov 21, 2026

    @shcheklein
    Member

    The repeated oauth2client.client message came from the old authentication library. On main, #395 uses google-auth and coordinates token refreshes for workers sharing a credential file. The PR verified that four processes sharing an expired cache made one refresh request and reused the rotated token. This change is pending release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions