# Changelog

## Score

- CAI 68 → 66 (-2.1)
- Rubric changed (rubric-2026.10.5 → rubric-2026.11.4) — scores are not directly comparable.

## Lenses

- Code Health 60 → 61 (+1.3)
- Architecture 97 → 96 (-0.6)
- Maturity 77 → 78 (+1.0)
- Readiness 74 → 63 (-11.9)
- Security 73 → 69 (-3.7)
- Performance 81 → 94 (+13.5)

## Resolved (121)

- Boundary-crossing change coupling: error.rs ↔ ShimSparkErrorConverter.scala (native/common/src/error.rs)
- Boundary-crossing change coupling: error.rs ↔ ShimSparkErrorConverter.scala (native/common/src/error.rs)
- Boundary-crossing change coupling: lib.rs ↔ NativeBase.java (native/core/src/lib.rs)
- ColumnarToRowContext::write_column_fixed_width (cognitive 17) (native/core/src/execution/columnar_to_row.rs)
- ColumnarToRowContext::write_column_fixed_width (cyclomatic 18) (native/core/src/execution/columnar_to_row.rs)
- CometDataWritingCommand.getSupportLevel (cognitive 19) (spark/src/main/scala/org/apache/comet/serde/operator/CometDataWritingCommand.scala)
- Duplicated block (10 lines × 2) (native/common/src/utf8.rs)
- Duplicated block (10 lines × 2) (native/shuffle/src/remote_schema.rs)
- Duplicated block (10 lines × 2) (native/spark-expr/src/array_funcs/array_contains.rs)
- Duplicated block (10 lines × 2) (spark/src/main/scala/org/apache/comet/codegen/CometBatchKernelCodegenInput.scala)
- Duplicated block (10 lines × 3) (spark/src/main/spark-3.5/org/apache/comet/serde/operator/CometMergeRows.scala)
- Duplicated block (11 lines × 2) (native/core/src/parquet/cast_column/variant.rs)
- Duplicated block (11 lines × 2) (native/spark-expr/src/agg_funcs/sum_int.rs)
- Duplicated block (11 lines × 2) (native/spark-expr/src/datetime_funcs/make_date.rs)
- Duplicated block (11 lines × 2) (spark/src/main/spark-3.4/org/apache/comet/shims/ShimCometSparkSessionExtensions.scala)
- Duplicated block (11 lines × 3) (native/spark-expr/src/agg_funcs/sum_int.rs)
- Duplicated block (11–14 lines × 3) (spark/src/main/scala/org/apache/spark/sql/comet/CometMetricNode.scala)
- Duplicated block (11–15 lines × 3) (spark/src/main/scala/org/apache/spark/sql/comet/CometMetricNode.scala)
- Duplicated block (12 lines × 2) (native/core/src/execution/columnar_to_row.rs)
- Duplicated block (12 lines × 2) (native/core/src/parquet/cast_column/variant.rs)
- …and 101 more

## New (315)

- Boundary-crossing change coupling: error.rs ↔ ShimSparkErrorConverter.scala (native/common/src/error.rs)
- Boundary-crossing change coupling: parquet_writer.rs ↔ CometNativeWriteExec.scala (native/core/src/execution/operators/parquet_writer.rs)
- CometIcebergNativeScan.hdfsEncryptionZoneFallback (cognitive 20) (spark/src/main/scala/org/apache/comet/serde/operator/CometIcebergNativeScan.scala)
- CometMakeDate.unsupportedParentReason (cognitive 32) (spark/src/main/scala/org/apache/comet/serde/datetime.scala)
- CometMakeDate.unsupportedParentReason (cyclomatic 20) (spark/src/main/scala/org/apache/comet/serde/datetime.scala)
- CometNativeShuffleWriter.writeInternal (cyclomatic 16) (spark/src/main/scala/org/apache/spark/sql/comet/execution/shuffle/CometNativeShuffleWriter.scala)
- CometScanRule.validateIcebergFileScanTasks (cyclomatic 17) (spark/src/main/scala/org/apache/comet/rules/CometScanRule.scala)
- CometWindowExec.convert (cognitive 17) (spark/src/main/scala/org/apache/spark/sql/comet/CometWindowExec.scala)
- Duplicated block (10 lines × 2) (native/common/src/utf8.rs)
- Duplicated block (10 lines × 2) (native/shuffle/src/remote_schema.rs)
- Duplicated block (10 lines × 2) (native/spark-expr/src/array_funcs/array_contains.rs)
- Duplicated block (11 lines × 2) (native/core/src/parquet/cast_column/variant.rs)
- Duplicated block (11 lines × 2) (native/shuffle/src/spark_unsafe/list.rs)
- Duplicated block (11 lines × 2) (native/spark-expr/src/agg_funcs/covariance.rs)
- Duplicated block (11 lines × 3) (native/spark-expr/src/agg_funcs/sum_int.rs)
- Duplicated block (12 lines × 2) (native/core/src/parquet/cast_column/variant.rs)
- Duplicated block (12 lines × 2) (native/spark-expr/src/agg_funcs/sum_int.rs)
- Duplicated block (12–13 lines × 2) (spark/src/main/spark-3.5/org/apache/spark/sql/comet/shims/ShimSparkErrorConverter.scala)
- Duplicated block (12–15 lines × 2) (native/core/src/execution/planner/window.rs)
- Duplicated block (12–19 lines × 2) (spark/src/main/spark-3.5/org/apache/spark/sql/comet/shims/ShimSparkErrorConverter.scala)
- …and 295 more

## Changes since last survey

- 97 commits — 62 feature/other, 35 fixes

## By area

- spark/src — 42 commits
- docs/source — 16 commits
- native/core — 7 commits
- native/spark-expr — 7 commits
- .github/workflows — 6 commits
- native/native-executors — 6 commits
- (root) — 4 commits
- native/shuffle — 2 commits
- spark/pom.xml — 2 commits
- .ai/skills — 1 commit
- contrib/delta — 1 commit
- dev/generate-release-docs.sh — 1 commit
- native/.cargo — 1 commit
- native/jni-bridge — 1 commit

## Notable commits

- fix: fix: align nested collection buffer nullability before spilling (#5903)
- fix: fix: apply the session's Comet configs in map stages of jobs outside a SQL execution (#7302)
- fix: fix: copy rows retained by identity broadcasts (#7260)
- fix: fix: declare required child distribution on aggregate, window and join operators (#7231)
- fix: fix: declare the input order of a native window over a sorted cache (#7256)
- fix: fix: decode invalid UTF-8 in nested Parquet BINARY values read as strings (#7106)
- fix: fix: don't treat planner-inserted casts as Parquet schema conversions (#7159)
- fix: fix: evaluate the right operand of AND, OR and IN only for rows the left side has not decided (#7326)
- fix: fix: fall back for brotli Iceberg writes when Hadoop's BrotliCodec cannot be loaded (#6996)
- fix: fix: fall back to Spark when the driver cannot use the ABFS authentication (#7221)
- fix: fix: give zstd the real destination capacity when compressing cached batches (#7307)
- fix: fix: include licenses for bundled lz4_flex and Zstandard (#7382)
- fix: fix: keep URL passwords and SAS queries out of Azure and object store errors, keys and logs (#7127)
- fix: fix: locate the test UDF library under Cargo's Rust 1.100 build layout (#7617)
- fix: fix: match SPARK-58821 and SPARK-60118 in sequence and string to decimal cast (#7171)
- fix: fix: merge all Bloom filter states and respect sliced offsets (#7173)
- fix: fix: pass the session batch size to the native FilterExec (#7280)
- fix: fix: pick the signed zero Spark ranks first in native percentile (#6928)
- fix: fix: point the external sort OOM error at Comet's memory settings (#7104)
- fix: fix: prefer Unsupported when combining nested cast support (#6501)
- …and 77 more

## Architecture

- Unchanged — 0 containers · 2 contexts · 0 edges
