Velocity Curve
SECSEC-01  ·  Security← The register

A key left in a code project's saved history kept running up charges for months, with nothing to cap them, until a student's bill reached 55,444.78 USD

Determination

A single API key published in one commit on GitHub stayed fully usable and billable for about three months, with no cap or reachable alert to surface the mounting charges until another developer warned the student account holder.

Incident

A student signed up for a cloud account using a student email address and worked inside the 300 USD of free credit. About 80 USD of that had gone on small learning exercises, leaving more than 220 USD. In early summer, an API key was pushed to GitHub inside a code project the account holder believed was private, where it sat in the project's saved history.

An invoice for 732 USD followed in June. It was not collected, because the payment card on file had already expired, and no notice of a failed payment reached the account holder's phone. A further invoice, for more than 31,000 USD, arrived in August, and another 21,000 USD in charges accumulated in early September. The student email address was not being read over the summer break.

In September, the developer who spotted the exposed key sent a notification through GitHub saying the key had been public for a long time and that others were using it. The account holder opened the account, found a total of 55,444.78 USD, and cancelled the key immediately. Over the 2 days that followed, more than 14,200 further attempts to use it were made by the unknown parties who used the key, at a 100 % failure rate.

The account holder contacted the cloud provider's billing support, filed a police report, and supplied usage logs, the GitHub links, screenshots and other documents, explaining that the card on file had expired. The case was reviewed and the answer was that the charges remained in effect: no cancellation, no changes. Notices arrived stating that if the balance was not paid within 10 days the debt would be transferred to a collections agency, with possible additional fees. In the account holder's country, the average daily income is around 15 USD.

Late in September, after further communication with the cloud provider's billing specialists, the case was reviewed again. The account holder wrote that "my case was reviewed again and the total outstanding balance has been completely waived". Whether any of the charges were ever collected is unknown.

Primary source — S1: https://www.reddit.com/r/googlecloud/comments/1noctxi/student_hit_with_a_5544478_google_cloud_billSnapshot

Cause of loss

  • A working key could keep adding to the bill without any ceiling, and the growing charges never stood out from an ordinary quiet accountEvery call a live key makes is billable, so the size of a loss is set by the ceiling on the account and by how quickly a warning lands somewhere a person is looking. With no ceiling in effect and the billing correspondence going to a mailbox that was not being read over the summer, the total could climb at machine speed for months before any human saw it.Technical name: uncapped billable credential
  • A key that reached the saved history of a code project stayed valid for months, with nothing cancelling it and nothing flagging that it could be readA code-sharing site keeps every earlier version of a project, so a key is still readable in the history behind the current files. A key that does not expire by itself stays exactly as usable as the day it was written until somebody cancels it, which is why the exposure ran from early summer until a stranger wrote in September.Technical name: long-lived secret in version history

Prevention

Put a ceiling on the account, and check that the alarm reaches a place you look

Ask the company running your cloud account, in writing, one question: when my account passes the limit I set, does paid use stop, or does it keep running and keep billing me? Keep the answer. Then put the email address and phone number you genuinely read onto the account, and keep the payment card details current, because a payment that fails quietly removes the one signal most people rely on. Treat free credit as a starting balance, never as a wall.

Assume anything that has ever been near a shared code project is public, and cancel on that basis

Cancel and replace every key that has ever sat in a code project, including projects you believe are private. Deleting the file is not enough; the site keeps the older versions behind it, and the key stays good until someone cancels it. Switch on your code host's secret scanning and its block-on-push setting for every project you own, and keep keys that can spend money off anything you are learning or experimenting in.

A leaked key costs you once; an account with no ceiling costs you every day nobody is looking.