Uploaded image for project: 'Cockpit'
  1. Cockpit
  2. COCKPIT-1288

Reduce traffic on Linode S3 buckets

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Done
    • Icon: Major Major
    • None
    • None
    • Automation and Tests
    • None
    • 25Q3 - July/02
    • Important

      Over the past year we've got increasingly ridiculous traffic on our Linode S3 buckets. Four days into the month we already got a warning:

      Your account, with the username cockpit, has exceeded the transfer quota notification threshold. It has used 1143GB of 1000GB or 114.3000%.

      So we've been paying increasing amounts of money for the excess traffic (8 Terabytes last month), and this is just a giant waste of resources.

      Keeping test logs public is important for contributors and generally as a FOSS project, so we don't (yet) want to hide them behind authentication.

      As a first step, we could try put an anubis proxy in front of them, and restrict access without a token to that IP. If that helps, it's good, otherwise we have to think about more dramatic measurs.

      But it could also be the VM image downloads – we just don't know, there are no per-bucket stats. For that the only reasonable thing is to make all of them private and require a token, and hand out tokens to outside contributors. Note that these have per-file ACLs, so we need to change them with a script.

              rhn-engineering-mpitt Martin Pitt
              rhn-engineering-mpitt Martin Pitt
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: