Timeout and Infinite retry in mutate_rows due to incorrect family name

[adelsaadeddin]

I have a simple script designed to import data into the Bigtable emulator for testing purposes. However, the execution of mutate_rows is taking an excessively long time and eventually times out.

Upon investigating and debugging the issue, I discovered that an unknown family_name is causing the package to enter an infinite retry loop, continuously retrying with sleep intervals, without throwing the actual exception.

Environment details

• OS type and version: macOS 14.2.1 (23C71)
• Python version: 3.11.9
• pip version: pip 24.0
• google-cloud-bigtable version: 2.23.1

Code example

def setup_bigtable():
    client = bigtable.Client(project=os.getenv('BIGTABLE_CACHE_PROJECT'), admin=True)
    instance = client.instance(os.getenv('BIGTABLE_CACHE_INSTANCE'))

    location = os.path.join(os.getenv('TEST_PATH'), 'fixtures/bigtable')
    subdir, _, files = next(os.walk(location))

    for file in files:
        table_name = os.path.splitext(file)[0]

        table = instance.table(table_name)
        if table.exists():
            table.delete()
        table.create()

        with open(f'{subdir}/{file}', 'rt') as fp:
            records = json.load(fp)

        rows = []
        for record in records:
            row = table.direct_row(record['key'])
            record.pop('key')

            for family, columns in record.items():
                for column, value in columns.items():
                    row.set_cell(family, column, value.encode())
                    rows.append(row)

        table.mutate_rows(rows)
    client.close()

The data will be:

families: 

['catalog']

rows:

[row_key: "sku#N10000019A"
mutations {
  set_cell {
    family_name: "catalog"
    column_qualifier: "product_title"
    timestamp_micros: -1
    value: "Some value"
  }
}
]

The response:

entries {
  status {
    code: 13
    message: "unknown family \"catalog\""
  }
}

In the code python-bigtable, the "unknown family \"catalog\"" (status=13, InternalServerError) error is considered retryable. Consequently, in api_core, the _BigtableRetryableError() exception is caught, causing the script to keep retrying with a sleep duration.