diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 342d7b0..8bb24be 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -1 +1,2 @@ -* @actions/actions-cache +* @justvanilla +* @inossidabile \ No newline at end of file diff --git a/README.md b/README.md index a98b4fb..9103ed3 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ -> This repo is a drop-in replacement for standard cache action to make it use AWS S3. You can replace cache calls with `uses: justvanilla/shared-gha-cache@s3` directly. Default behavior will be identical. To make task use S3 bucket instead, provide the following inputs: +> This repo is a drop-in replacement for standard cache action to make it use AWS S3. You can replace cache calls with `uses: justvanilla/shared-gha-cache-s3@v3` directly. Default behavior will be identical. To make task use S3 bucket instead, provide the following inputs: > * **aws-region**: us-east-1 > * **aws-bucket**: the-bucketest-bucket > * **aws-access-key-id**: key id for account having RW access to the bucket @@ -109,7 +109,7 @@ jobs: - name: Cache Primes id: cache-primes - uses: justvanilla/shared-gha-cache@s3 + uses: justvanilla/shared-gha-cache-s3@v3 with: path: prime-numbers key: ${{ runner.os }}-primes @@ -205,7 +205,7 @@ A cache key can include any of the contexts, functions, literals, and operators For example, using the [`hashFiles`](https://docs.github.com/en/actions/learn-github-actions/expressions#hashfiles) function allows you to create a new cache when dependencies change. ```yaml - - uses: justvanilla/shared-gha-cache@s3 + - uses: justvanilla/shared-gha-cache-s3@v3 with: path: | path/to/dependencies @@ -227,7 +227,7 @@ Additionally, you can use arbitrary command output in a cache key, such as a dat echo "date=$(/bin/date -u "+%Y%m%d")" >> $GITHUB_OUTPUT shell: bash - - uses: justvanilla/shared-gha-cache@s3 + - uses: justvanilla/shared-gha-cache-s3@v3 with: path: path/to/dependencies key: ${{ runner.os }}-${{ steps.get-date.outputs.date }}-${{ hashFiles('**/lockfiles') }} @@ -253,11 +253,15 @@ Example: steps: - uses: actions/checkout@v3 - - uses: justvanilla/shared-gha-cache@s3 + - uses: justvanilla/shared-gha-cache-s3@v3 id: cache with: path: path/to/dependencies key: ${{ runner.os }}-${{ hashFiles('**/lockfiles') }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} - name: Install Dependencies if: steps.cache.outputs.cache-hit != 'true' @@ -285,10 +289,14 @@ jobs: - name: Cache Primes id: cache-primes - uses: justvanilla/shared-gha-cache@s3 + uses: justvanilla/shared-gha-cache-s3@v3 with: path: prime-numbers key: primes + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} - name: Generate Prime Numbers if: steps.cache-primes.outputs.cache-hit != 'true' @@ -296,10 +304,14 @@ jobs: - name: Cache Numbers id: cache-numbers - uses: justvanilla/shared-gha-cache@s3 + uses: justvanilla/shared-gha-cache-s3@v3 with: path: numbers key: primes + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} - name: Generate Numbers if: steps.cache-numbers.outputs.cache-hit != 'true' @@ -312,10 +324,14 @@ jobs: - name: Cache Primes id: cache-primes - uses: justvanilla/shared-gha-cache@s3 + uses: justvanilla/shared-gha-cache-s3@v3 with: path: prime-numbers key: primes + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} - name: Generate Prime Numbers if: steps.cache-primes.outputs.cache-hit != 'true' diff --git a/action.yml b/action.yml index 0f29d81..a701f70 100644 --- a/action.yml +++ b/action.yml @@ -1,6 +1,6 @@ name: 'Cache (AWS S3)' description: 'Cache artifacts like dependencies and build outputs to improve workflow execution time stored in S3' -author: 'GitHub' +author: 'Github, Justvanilla' inputs: path: description: 'A list of files, directories, and wildcard patterns to cache and restore' diff --git a/caching-strategies.md b/caching-strategies.md index 47946c3..267000f 100644 --- a/caching-strategies.md +++ b/caching-strategies.md @@ -12,9 +12,13 @@ This document lists some of the strategies (and example workflows if possible) w jobs: build: runs-on: ubuntu-latest - - uses: actions/cache@v3 + - uses: justvanilla/shared-gha-cache-s3@v3 with: key: ${{ some-metadata }}-cache + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` In your workflows, you can use different strategies to name your key depending on your use case so that the cache is scoped appropriately for your need. For example, you can have cache specific to OS, or based on the lockfile or commit SHA or even workflow run. @@ -24,12 +28,16 @@ In your workflows, you can use different strategies to name your key depending o One of the most common use case is to use hash for lockfile as key. This way, same cache will be restored for a lockfile until there's a change in dependencies listed in lockfile. ```yaml - - uses: actions/cache@v3 + - uses: justvanilla/shared-gha-cache-s3@v3 with: path: | path/to/dependencies some/other/dependencies key: cache-${{ hashFiles('**/lockfiles') }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ### Using restore keys to download the closest matching cache @@ -37,7 +45,7 @@ One of the most common use case is to use hash for lockfile as key. This way, sa If cache is not found matching the primary key, restore keys can be used to download the closest matching cache that was recently created. This ensures that the build/install step will need to additionally fetch just a handful of newer dependencies, and hence saving build time. ```yaml - - uses: actions/cache@v3 + - uses: justvanilla/shared-gha-cache-s3@v3 with: path: | path/to/dependencies @@ -45,6 +53,10 @@ If cache is not found matching the primary key, restore keys can be used to down key: cache-npm-${{ hashFiles('**/lockfiles') }} restore-keys: | cache-npm- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` The restore keys can be provided as a complete name, or a prefix, read more [here](https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#matching-a-cache-key) on how a cache key is matched using restore keys. @@ -54,12 +66,16 @@ The restore keys can be provided as a complete name, or a prefix, read more [her In case of workflows with matrix running for multiple Operating Systems, the caches can be stored separately for each of them. This can be used in combination with hashfiles in case multiple caches are being generated per OS. ```yaml - - uses: actions/cache@v3 + - uses: justvanilla/shared-gha-cache-s3@v3 with: path: | path/to/dependencies some/other/dependencies key: ${{ runner.os }}-cache + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ### Creating a short lived cache @@ -73,12 +89,16 @@ Caches scoped to the particular workflow run id or run attempt can be stored and On similar lines, commit sha can be used to create a very specialized and short lived cache. ```yaml - - uses: actions/cache@v3 + - uses: justvanilla/shared-gha-cache-s3@v3 with: path: | path/to/dependencies some/other/dependencies key: cache-${{ github.sha }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ### Using multiple factors while forming a key depening on the need @@ -86,12 +106,16 @@ On similar lines, commit sha can be used to create a very specialized and short Cache key can be formed by combination of more than one metadata, evaluated info. ```yaml - - uses: actions/cache@v3 + - uses: justvanilla/shared-gha-cache-s3@v3 with: path: | path/to/dependencies some/other/dependencies key: ${{ runner.os }}-${{ hashFiles('**/lockfiles') }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` The [GitHub Context](https://docs.github.com/en/actions/learn-github-actions/contexts#github-context) can be used to create keys using the workflows metadata. @@ -148,7 +172,7 @@ In case you are using a centralized job to create and save your cache that can b steps: - uses: actions/checkout@v3 - - uses: actions/cache/restore@v3 + - uses: justvanilla/shared-gha-cache-s3/restore@v3 id: cache with: path: path/to/dependencies @@ -173,7 +197,7 @@ You can use the output of this action to exit the workflow on cache miss. This w steps: - uses: actions/checkout@v3 - - uses: actions/cache/restore@v3 + - uses: justvanilla/shared-gha-cache-s3/restore@v3 id: cache with: path: path/to/dependencies @@ -194,22 +218,30 @@ steps: If you want to avoid re-computing the cache key again in `save` action, the outputs from `restore` action can be used as input to the `save` action. ```yaml - - uses: actions/cache/restore@v3 + - uses: justvanilla/shared-gha-cache-s3/restore@v3 id: restore-cache with: path: | path/to/dependencies some/other/dependencies key: ${{ runner.os }}-${{ hashFiles('**/lockfiles') }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} . . . - - uses: actions/cache/save@v3 + - uses: justvanilla/shared-gha-cache-s3/save@v3 with: path: | path/to/dependencies some/other/dependencies key: ${{ steps.restore-cache.outputs.cache-primary-key }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ### Re-evaluate cache key while saving cache @@ -219,33 +251,45 @@ On the other hand, the key can also be explicitly re-computed while executing th Let's say we have a restore step that computes key at runtime ```yaml -uses: actions/cache/restore@v3 +uses: justvanilla/shared-gha-cache-s3/restore@v3 id: restore-cache with: key: cache-${{ hashFiles('**/lockfiles') }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` Case 1: Where an user would want to reuse the key as it is ```yaml -uses: actions/cache/save@v3 +uses: justvanilla/shared-gha-cache-s3/save@v3 with: key: ${{ steps.restore-cache.outputs.cache-primary-key }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` Case 2: Where the user would want to re-evaluate the key ```yaml -uses: actions/cache/save@v3 +uses: justvanilla/shared-gha-cache-s3/save@v3 with: key: npm-cache-${{hashfiles(package-lock.json)}} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ### Saving cache even if the build fails -There can be cases where a cache should be saved even if the build job fails. For example, a job can fail due to flaky tests but the caches can still be re-used. You can use `actions/cache/save` action to save the cache by using `if: always()` condition. +There can be cases where a cache should be saved even if the build job fails. For example, a job can fail due to flaky tests but the caches can still be re-used. You can use `justvanilla/shared-gha-cache-s3/save` action to save the cache by using `if: always()` condition. -Similarly, `actions/cache/save` action can be conditionally used based on the output of the previous steps. This way you get more control on when to save the cache. +Similarly, `justvanilla/shared-gha-cache-s3/save` action can be conditionally used based on the output of the previous steps. This way you get more control on when to save the cache. ```yaml steps: @@ -255,16 +299,20 @@ steps: . - name: Build run: /build.sh - - uses: actions/cache/save@v3 + - uses: justvanilla/shared-gha-cache-s3/save@v3 if: always() // or any other condition to invoke the save action with: path: path/to/dependencies key: ${{ runner.os }}-${{ hashFiles('**/lockfiles') }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ### Saving cache once and reusing in multiple workflows -In case of multi-module projects, where the built artifact of one project needs to be reused in subsequent child modules, the need of rebuilding the parent module again and again with every build can be eliminated. The `actions/cache` or `actions/cache/save` action can be used to build and save the parent module artifact once, and restored multiple times while building the child modules. +In case of multi-module projects, where the built artifact of one project needs to be reused in subsequent child modules, the need of rebuilding the parent module again and again with every build can be eliminated. The `justvanilla/shared-gha-cache-s3` or `justvanilla/shared-gha-cache-s3/save` action can be used to build and save the parent module artifact once, and restored multiple times while building the child modules. #### Step 1 - Build the parent module and save it @@ -275,11 +323,15 @@ steps: - name: Build run: ./build-parent-module.sh - - uses: actions/cache/save@v3 + - uses: justvanilla/shared-gha-cache-s3/save@v3 id: cache with: path: path/to/dependencies key: ${{ runner.os }}-${{ hashFiles('**/lockfiles') }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` #### Step 2 - Restore the built artifact from cache using the same key and path @@ -288,11 +340,15 @@ steps: steps: - uses: actions/checkout@v3 - - uses: actions/cache/restore@v3 + - uses: justvanilla/shared-gha-cache-s3/restore@v3 id: cache with: path: path/to/dependencies key: ${{ runner.os }}-${{ hashFiles('**/lockfiles') }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} - name: Install Dependencies if: steps.cache.outputs.cache-hit != 'true' diff --git a/examples.md b/examples.md index 1bcde91..fcdbe9f 100644 --- a/examples.md +++ b/examples.md @@ -46,7 +46,7 @@ Using [NuGet lock files](https://docs.microsoft.com/nuget/consume-packages/package-references-in-project-files#locking-dependencies): ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: ~/.nuget/packages key: ${{ runner.os }}-nuget-${{ hashFiles('**/packages.lock.json') }} @@ -55,10 +55,10 @@ Using [NuGet lock files](https://docs.microsoft.com/nuget/consume-packages/packa ``` Depending on the environment, huge packages might be pre-installed in the global cache folder. -With `actions/cache@v3` you can now exclude unwanted packages with [exclude pattern](https://github.com/actions/toolkit/tree/main/packages/glob#exclude-patterns) +With `justvanilla/shared-gha-cache-s3@v3` you can now exclude unwanted packages with [exclude pattern](https://github.com/actions/toolkit/tree/main/packages/glob#exclude-patterns) ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: | ~/.nuget/packages @@ -75,24 +75,32 @@ Or you could move the cache folder like below. env: NUGET_PACKAGES: ${{ github.workspace }}/.nuget/packages steps: - - uses: actions/cache@v3 + - uses: justvanilla/shared-gha-cache-s3@v3 with: path: ${{ github.workspace }}/.nuget/packages key: ${{ runner.os }}-nuget-${{ hashFiles('**/packages.lock.json') }} restore-keys: | ${{ runner.os }}-nuget- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## Clojure - Lein Deps ```yaml - name: Cache lein project dependencies - uses: actions/cache@v3 + uses: justvanilla/shared-gha-cache-s3@v3 with: path: ~/.m2/repository key: ${{ runner.os }}-clojure-${{ hashFiles('**/project.clj') }} restore-keys: | ${{ runner.os }}-clojure + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` @@ -101,23 +109,31 @@ steps: ### POSIX ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: ~/.dub key: ${{ runner.os }}-dub-${{ hashFiles('**/dub.selections.json') }} restore-keys: | ${{ runner.os }}-dub- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ### Windows ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: ~\AppData\Local\dub key: ${{ runner.os }}-dub-${{ hashFiles('**/dub.selections.json') }} restore-keys: | ${{ runner.os }}-dub- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## Deno @@ -125,40 +141,52 @@ steps: ### Linux ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: | ~/.deno ~/.cache/deno key: ${{ runner.os }}-deno-${{ hashFiles('**/deps.ts') }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ### macOS ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: | ~/.deno ~/Library/Caches/deno key: ${{ runner.os }}-deno-${{ hashFiles('**/deps.ts') }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ### Windows ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: | ~\.deno ~\AppData\Local\deno key: ${{ runner.os }}-deno-${{ hashFiles('**/deps.ts') }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## Elixir - Mix ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: | deps @@ -166,6 +194,10 @@ steps: key: ${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }} restore-keys: | ${{ runner.os }}-mix- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## Erlang - Rebar3 @@ -178,6 +210,10 @@ steps: key: ${{ runner.os }}-erlang-${{ env.OTP_VERSION }}-${{ hashFiles('**/*rebar.lock') }} restore-keys: | ${{ runner.os }}-erlang-${{ env.OTP_VERSION }}- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## Go - Modules @@ -185,7 +221,7 @@ steps: ### Linux ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: | ~/.cache/go-build @@ -193,12 +229,16 @@ steps: key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }} restore-keys: | ${{ runner.os }}-go- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ### macOS ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: | ~/Library/Caches/go-build @@ -206,12 +246,16 @@ steps: key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }} restore-keys: | ${{ runner.os }}-go- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ### Windows ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: | ~\AppData\Local\go-build @@ -219,6 +263,10 @@ steps: key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }} restore-keys: | ${{ runner.os }}-go- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## Haskell - Cabal @@ -227,7 +275,7 @@ We cache the elements of the Cabal store separately, as the entirety of `~/.caba ```yaml - name: Cache ~/.cabal/packages, ~/.cabal/store and dist-newstyle - uses: actions/cache@v3 + uses: justvanilla/shared-gha-cache-s3@v3 with: path: | ~/.cabal/packages @@ -235,6 +283,10 @@ We cache the elements of the Cabal store separately, as the entirety of `~/.caba dist-newstyle key: ${{ runner.os }}-${{ matrix.ghc }}-${{ hashFiles('**/*.cabal', '**/cabal.project', '**/cabal.project.freeze') }} restore-keys: ${{ runner.os }}-${{ matrix.ghc }}- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## Haskell - Stack @@ -242,26 +294,34 @@ We cache the elements of the Cabal store separately, as the entirety of `~/.caba ### Linux or macOS ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 name: Cache ~/.stack with: path: ~/.stack key: ${{ runner.os }}-stack-global-${{ hashFiles('stack.yaml') }}-${{ hashFiles('package.yaml') }} restore-keys: | ${{ runner.os }}-stack-global- -- uses: actions/cache@v3 + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} +- uses: justvanilla/shared-gha-cache-s3@v3 name: Cache .stack-work with: path: .stack-work key: ${{ runner.os }}-stack-work-${{ hashFiles('stack.yaml') }}-${{ hashFiles('package.yaml') }}-${{ hashFiles('**/*.hs') }} restore-keys: | ${{ runner.os }}-stack-work- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ### Windows ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 name: Cache %APPDATA%\stack %LOCALAPPDATA%\Programs\stack with: path: | @@ -270,13 +330,21 @@ We cache the elements of the Cabal store separately, as the entirety of `~/.caba key: ${{ runner.os }}-stack-global-${{ hashFiles('stack.yaml') }}-${{ hashFiles('package.yaml') }} restore-keys: | ${{ runner.os }}-stack-global- -- uses: actions/cache@v3 + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} +- uses: justvanilla/shared-gha-cache-s3@v3 name: Cache .stack-work with: path: .stack-work key: ${{ runner.os }}-stack-work-${{ hashFiles('stack.yaml') }}-${{ hashFiles('package.yaml') }}-${{ hashFiles('**/*.hs') }} restore-keys: | ${{ runner.os }}-stack-work- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## Java - Gradle @@ -284,7 +352,7 @@ We cache the elements of the Cabal store separately, as the entirety of `~/.caba > **Note** Ensure no Gradle daemons are running anymore when your workflow completes. Creating the cache package might fail due to locks being held by Gradle. Refer to the [Gradle Daemon documentation](https://docs.gradle.org/current/userguide/gradle_daemon.html) on how to disable or stop the Gradle Daemons. ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: | ~/.gradle/caches @@ -292,18 +360,26 @@ We cache the elements of the Cabal store separately, as the entirety of `~/.caba key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }} restore-keys: | ${{ runner.os }}-gradle- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## Java - Maven ```yaml - name: Cache local Maven repository - uses: actions/cache@v3 + uses: justvanilla/shared-gha-cache-s3@v3 with: path: ~/.m2/repository key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }} restore-keys: | ${{ runner.os }}-maven- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## Node - npm @@ -334,23 +410,31 @@ After [deprecation](https://github.blog/changelog/2022-10-11-github-actions-depr `Get npm cache directory` step can then be used with `actions/cache` as shown below ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 id: npm-cache # use this to check for `cache-hit` ==> if: steps.npm-cache.outputs.cache-hit != 'true' with: path: ${{ steps.npm-cache-dir.outputs.dir }} key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }} restore-keys: | ${{ runner.os }}-node- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## Node - Lerna ```yaml - name: restore lerna - uses: actions/cache@v3 + uses: justvanilla/shared-gha-cache-s3@v3 with: path: '**/node_modules' key: ${{ runner.os }}-${{ hashFiles('**/yarn.lock') }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## Node - Yarn @@ -361,13 +445,17 @@ The yarn cache directory will depend on your operating system and version of `ya id: yarn-cache-dir-path run: echo "dir=$(yarn cache dir)" >> $GITHUB_OUTPUT -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 id: yarn-cache # use this to check for `cache-hit` (`steps.yarn-cache.outputs.cache-hit != 'true'`) with: path: ${{ steps.yarn-cache-dir-path.outputs.dir }} key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }} restore-keys: | ${{ runner.os }}-yarn- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## Node - Yarn 2 @@ -379,13 +467,17 @@ The yarn 2 cache directory will depend on your config. See https://yarnpkg.com/c id: yarn-cache-dir-path run: echo "dir=$(yarn config get cacheFolder)" >> $GITHUB_OUTPUT -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 id: yarn-cache # use this to check for `cache-hit` (`steps.yarn-cache.outputs.cache-hit != 'true'`) with: path: ${{ steps.yarn-cache-dir-path.outputs.dir }} key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }} restore-keys: | ${{ runner.os }}-yarn- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## OCaml/Reason - esy @@ -394,12 +486,16 @@ Esy allows you to export built dependencies and import pre-built dependencies. ```yaml - name: Restore Cache id: restore-cache - uses: actions/cache@v3 + uses: justvanilla/shared-gha-cache-s3@v3 with: path: _export key: ${{ runner.os }}-esy-${{ hashFiles('esy.lock/index.json') }} restore-keys: | ${{ runner.os }}-esy- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} - name: Esy install run: 'esy install' - name: Import Cache @@ -423,12 +519,16 @@ Esy allows you to export built dependencies and import pre-built dependencies. id: composer-cache run: | echo "dir=$(composer config cache-files-dir)" >> $GITHUB_OUTPUT -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: ${{ steps.composer-cache.outputs.dir }} key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }} restore-keys: | ${{ runner.os }}-composer- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## Python - pip @@ -444,12 +544,16 @@ Locations: ### Simple example ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: ~/.cache/pip key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }} restore-keys: | ${{ runner.os }}-pip- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` Replace `~/.cache/pip` with the correct `path` if not using Ubuntu. @@ -457,29 +561,41 @@ Replace `~/.cache/pip` with the correct `path` if not using Ubuntu. ### Multiple OS's in a workflow ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 if: startsWith(runner.os, 'Linux') with: path: ~/.cache/pip key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }} restore-keys: | ${{ runner.os }}-pip- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 if: startsWith(runner.os, 'macOS') with: path: ~/Library/Caches/pip key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }} restore-keys: | ${{ runner.os }}-pip- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 if: startsWith(runner.os, 'Windows') with: path: ~\AppData\Local\pip\Cache key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }} restore-keys: | ${{ runner.os }}-pip- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ### Multiple OS's in a workflow with a matrix @@ -499,12 +615,16 @@ jobs: - os: windows-latest path: ~\AppData\Local\pip\Cache steps: - - uses: actions/cache@v3 + - uses: justvanilla/shared-gha-cache-s3@v3 with: path: ${{ matrix.path }} key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }} restore-keys: | - ${{ runner.os }}-pip- + ${{ runner.os }}-pip- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ### Using pip to get cache location @@ -517,12 +637,16 @@ jobs: echo "dir=$(pip cache dir)" >> $GITHUB_OUTPUT - name: pip cache - uses: actions/cache@v3 + uses: justvanilla/shared-gha-cache-s3@v3 with: path: ${{ steps.pip-cache.outputs.dir }} key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }} restore-keys: | ${{ runner.os }}-pip- + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## Python - pipenv @@ -535,10 +659,14 @@ jobs: ⋮ -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: ~/.local/share/virtualenvs key: ${{ runner.os }}-python-${{ steps.setup-python.outputs.python-version }}-pipenv-${{ hashFiles('Pipfile.lock') }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` ## R - renv @@ -562,7 +690,7 @@ For renv, the cache directory will vary by OS. The `RENV_PATHS_ROOT` environment cat("##[set-output name=r-version;]", R.Version()$version.string, sep = "") shell: Rscript {0} - name: Restore Renv package cache - uses: actions/cache@v3 + uses: justvanilla/shared-gha-cache-s3@v3 with: path: ${{ env.RENV_PATHS_ROOT }} key: ${{ steps.get-version.outputs.os-version }}-${{ steps.get-version.outputs.r-version }}-${{ inputs.cache-version }}-${{ hashFiles('renv.lock') }} @@ -588,7 +716,7 @@ whenever possible: ## Rust - Cargo ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: | ~/.cargo/bin/ @@ -603,7 +731,7 @@ whenever possible: ```yaml - name: Cache SBT - uses: actions/cache@v3 + uses: justvanilla/shared-gha-cache-s3@v3 with: path: | ~/.ivy2/cache @@ -614,7 +742,7 @@ whenever possible: ## Swift, Objective-C - Carthage ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: Carthage key: ${{ runner.os }}-carthage-${{ hashFiles('**/Cartfile.resolved') }} @@ -625,7 +753,7 @@ whenever possible: ## Swift, Objective-C - CocoaPods ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: Pods key: ${{ runner.os }}-pods-${{ hashFiles('**/Podfile.lock') }} @@ -636,7 +764,7 @@ whenever possible: ## Swift - Swift Package Manager ```yaml -- uses: actions/cache@v3 +- uses: justvanilla/shared-gha-cache-s3@v3 with: path: .build key: ${{ runner.os }}-spm-${{ hashFiles('**/Package.resolved') }} @@ -651,7 +779,7 @@ env: MINT_PATH: .mint/lib MINT_LINK_PATH: .mint/bin steps: - - uses: actions/cache@v3 + - uses: justvanilla/shared-gha-cache-s3@v3 with: path: .mint key: ${{ runner.os }}-mint-${{ hashFiles('**/Mintfile') }} @@ -667,7 +795,7 @@ steps: ```yaml - name: Cache Bazel - uses: actions/cache@v3 + uses: justvanilla/shared-gha-cache-s3@v3 with: path: | ~/.cache/bazel @@ -681,7 +809,7 @@ steps: ```yaml - name: Cache Bazel - uses: actions/cache@v3 + uses: justvanilla/shared-gha-cache-s3@v3 with: path: | /private/var/tmp/_bazel_runner/ diff --git a/restore/action.yml b/restore/action.yml index f3384ad..8d10d01 100644 --- a/restore/action.yml +++ b/restore/action.yml @@ -1,6 +1,6 @@ name: 'Restore Cache (AWS S3)' description: 'Restore Cache artifacts stored in S3 like dependencies and build outputs to improve workflow execution time' -author: 'GitHub' +author: 'GitHub, Justvanilla' inputs: path: description: 'A list of files, directories, and wildcard patterns to restore' diff --git a/save/action.yml b/save/action.yml index 467d061..209fdce 100644 --- a/save/action.yml +++ b/save/action.yml @@ -1,6 +1,6 @@ name: 'Save a cache (AWS S3)' description: 'Save Cache artifacts in S3 like dependencies and build outputs to improve workflow execution time' -author: 'GitHub' +author: 'Github, Justvanilla' inputs: path: description: 'A list of files, directories, and wildcard patterns to cache' diff --git a/tips-and-workarounds.md b/tips-and-workarounds.md index 125fa5f..8410600 100644 --- a/tips-and-workarounds.md +++ b/tips-and-workarounds.md @@ -12,12 +12,16 @@ A cache today is immutable and cannot be updated. But some use cases require the ```yaml - name: update cache on every commit - uses: actions/cache@v3 + uses: justvanilla/shared-gha-cache-s3@v3 with: path: prime-numbers key: primes-${{ runner.os }}-${{ github.run_id }} # Can use time based key as well restore-keys: | primes-${{ runner.os }} + aws-region: ${{ secrets.CACHE_AWS_REGION }} + aws-bucket: ${{ secrets.CACHE_AWS_BUCKET }} + aws-access-key-id: ${{ secrets.CACHE_AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.CACHE_AWS_SECRET_ACCESS_KEY }} ``` Please note that this will create a new cache on every run and hence will consume the cache [quota](./README.md#cache-limits).