Add package resource file sizes to PackageRevision interface#954
Add package resource file sizes to PackageRevision interface#954JamesMcDermott wants to merge 19 commits into
Conversation
✅ Deploy Preview for porch ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
0830aa2 to
9a4dacc
Compare
|
Issue that it fixes is incorrect. |
Corrected - good spot, thanks. |
29b7a4f to
0be6e34
Compare
efiacor
left a comment
There was a problem hiding this comment.
Looks good overall.
I think we need to add some size checks for clone and edit e2e flows.
Maybe using assert.Greater() to avoid having to calculate the exact sizes.
45c6fe0 to
be9f82a
Compare
JamesMcDermott
left a comment
There was a problem hiding this comment.
Looks good overall. I think we need to add some size checks for clone and edit e2e flows. Maybe using assert.Greater() to avoid having to calculate the exact sizes.
Thanks - I've added the checks in the clone, edit and upgrade tests - the edit one also has a resources-update flow, so good call - it let me do a before/after comparison.
Not sure assert.Greater() would have any value given the adjustment I put in to avoid having exact sizes hardcoded in the tests...
|
@JamesMcDermott please mark any closed off comments (especially the CoPilot ones) as resolved if they are resolved. |
40fb113 to
fa7e6d4
Compare
|
2c701d9 to
eca0bf7
Compare
eca0bf7 to
20e4aa2
Compare
There was a problem hiding this comment.
Pull request overview
Adds a new prrSizeBytes status field on PackageRevision to expose the total size (bytes) of a package revision’s resource contents, with DB schema + migrations to persist/backfill the value and E2E/docs updates to validate and document the new field.
Changes:
- Add
PrrSizeBytestoPackageRevisionStatusacross API versions (v1alpha1 + v1alpha2), including conversion + OpenAPI + CRD schema. - Persist and read the value via DB cache (
resources_sizecolumn) and populate it during sync/close-draft flows. - Update E2E tests and documentation to surface and validate the new status field.
Reviewed changes
Copilot reviewed 25 out of 27 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| test/e2e/suiteutils/suite.go | Adds DB-cache detection flag and (problematic) import path changes. |
| test/e2e/api/rpkg_lifecycle_test.go | Validates revision and adds package resource size validation. |
| test/e2e/api/rpkg_init_test.go | Adds helper to validate PrrSizeBytes and updates imports. |
| test/e2e/api/rpkg_edit_test.go | Adds assertions around size changes when resources are updated. |
| test/e2e/api/rpkg_clone_test.go | Adds size validation after clone/upgrade flows. |
| test/e2e/api/repository_test.go | Adds size validation during list checks and updates imports. |
| pkg/cache/dbcache/dbreposync.go | Computes and stores resources size during external repo sync caching. |
| pkg/cache/dbcache/dbrepository.go | Computes resources size on draft close before persisting. |
| pkg/cache/dbcache/dbpackagerevisionsql.go | Wires resources_size into SELECT/INSERT/UPDATE SQL and scanning. |
| pkg/cache/dbcache/dbpackagerevision.go | Exposes DB-cached size via PackageRevisionStatus.PrrSizeBytes. |
| docs/content/en/docs/7_cli_api/api-ref.md | Documents the new prrSizeBytes status field. |
| docs/content/en/docs/5_architecture_and_components/package-cache/db-cache.md | Documents DB-cache behavior including resource size summing on close. |
| docs/content/en/docs/4_tutorials_and_how-tos/working_with_package_revisions/inspecting-packages.md | Minor formatting adjustment near output explanation. |
| docs/content/en/docs/2_concepts/package-revisions.md | Fixes YAML indentation and updates repo branch example. |
| deployments/porch/3-porch-postgres-bundle.yaml | Adds resources_size column to packaged Postgres schema. |
| controllers/packagevariants/pkg/controllers/packagevariant/packagevariant_controller-with-workspacename_test.go | Updates copyright header year. |
| api/sql/porch-db.sql | Adds resources_size column to base schema. |
| api/sql/porch-db-1.6.0-1.5.9.sql | Adds downgrade script to drop resources_size. |
| api/sql/porch-db-1.5.9-1.6.0.sql | Adds upgrade script to add/backfill resources_size. |
| api/sql/porch-db-1.5.9-1.5.10.sql | Adds upgrade script to add/backfill resources_size. |
| api/sql/porch-db-1.5.10-1.5.9.sql | Adds downgrade script to drop resources_size. |
| api/porch/v1alpha2/porch.kpt.dev_packagerevisions.yaml | Updates CRD schema docs and adds prrSizeBytes. |
| api/porch/v1alpha2/packagerevision_types.go | Adds PrrSizeBytes to v1alpha2 status type. |
| api/porch/v1alpha1/zz_generated.conversion.go | Wires PrrSizeBytes through generated conversions. |
| api/porch/v1alpha1/types.go | Adds PrrSizeBytes to v1alpha1 status type and fixes comments. |
| api/porch/types.go | Adds PrrSizeBytes to internal API status type and fixes comments. |
| api/generated/openapi/zz_generated.openapi.go | Updates generated OpenAPI schema to include prrSizeBytes. |
Files not reviewed (1)
- api/porch/v1alpha1/zz_generated.conversion.go: Language not supported
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 25 out of 27 changed files in this pull request and generated 6 comments.
Files not reviewed (1)
- api/porch/v1alpha1/zz_generated.conversion.go: Language not supported
Comments suppressed due to low confidence (1)
test/e2e/suiteutils/suite.go:167
- The body of
checkIfUsingDBCachewraps a trivial expression in an immediately-invoked anonymous function. This can be simplified to a direct assignment:t.UsingDBCache = os.Getenv("DB_CACHE") != "". The IIFE adds no value and obscures the logic.
func (t *TestSuite) checkIfUsingDBCache() {
t.UsingDBCache = func() bool {
return os.Getenv("DB_CACHE") != ""
}()
}
- on ClosePackageRevisionDraft:
- add up individual sizes of PackageRevisionResources
- store total as a separate metadata field in package_revisions
table
- on get/list PackageRevision, include total resources size in status
field of returned API object
kptdev#811
Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>
- document new API field `prrSizeOnDisk` - include new "SIZE ON DISK" table column in `rpkg get` output examples Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>
Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>
- fix E2E tests by excluding package-size testing from E2E tests for CR cache
- new mechanism in test suite to detect DB cache when in use
- can be used in test cases to exclude/include portions depending on cache
- remove "Size on Disk" column from `kubectl get packagerevisions` table
- agreed not essential information, may set precedent to overload table output
- also removes need for updates to E2E CLI tests
Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>
Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>
Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>
Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>
- also remove restriction that meant size didn't get updated on PackageRevisionResources update Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>
Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>
Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>
Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>
Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>
Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>
Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>
Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>
Signed-off-by: ezmcdja <james.j.mcdermott@ericsson.com>
Signed-off-by: ezmcdja <james.j.mcdermott@ericsson.com>
05a35f5 to
e76c342
Compare
Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>
Signed-off-by: James McDermott <james.j.mcdermott@ericsson.com>



Description
prrSizeBytesfield toPackageRevisionStatusin bothv1alpha1(existing) andv1alpha2(in-progress CRD migration) APIsPackageRevisions, the total resources size is included in thestatusfield of the returned API objectresources_sizeINTEGER column to thepackage_revisionsdatabase tableresources_sizecolumnresources_sizeprrSizeBytesstatus fieldRelated Issue(s)
Type of Change
Checklist
AI Disclosure
Examples: