Release Notes - Beam - Version 2.33.0 - HTML format

Sub-task

  • [BEAM-12635] - Beam Dependency Update Request: org.apache.commons:commons-compress

Bug

  • [BEAM-6374] - "elements added" for input and output collections is always empty
  • [BEAM-6516] - Failed to advance source: org.apache.beam.sdk.io.rabbitmq.RabbitMqIO
  • [BEAM-7195] - BigQuery - 404 errors for 'table not found' when using dynamic destinations - sometimes, new table fails to get created
  • [BEAM-12445] - Move Python's BigQuery streaming insert sink to the new BigQuery api client
  • [BEAM-12479] - UnsupportedOperationException when reading from BigQuery tables and converting TableRows to Beam Rows
  • [BEAM-12573] - Bounded Source Reader DoFn PR #13154 causes some pipelines to fail with a PicklingError
  • [BEAM-12596] - Ensure that size reported by an SDF is non-negative
  • [BEAM-12626] - Side inputs with Session windows are not populated
  • [BEAM-12628] - AvroCoder changed underlying String class for SpecificRecords
  • [BEAM-12639] - Go SDK SplitAndSizeRestrictions should allow for an empty split
  • [BEAM-12669] - UpdateDestinationSchema PTransform does not respect source format
  • [BEAM-12699] - Several streaming tests failing - PostCommit_Java_VR_Dataflow_V2_Streaming, Python PreCommit,
  • [BEAM-12702] - [Go SDK] Post job metrics querying broken for dataflow
  • [BEAM-12703] - [Go SDK] Universal runner shouldn't require tentative or committed metrics
  • [BEAM-12713] - beam_PostCommit_NightlySnapshot runMobileGamingJavaDataflow failing
  • [BEAM-12715] - SnowflakeWrite fails in batch mode when the number of shards is > 1000
  • [BEAM-12726] - Release_Build failing with :sdks:go:test:load:resolveBuildDependencies failure
  • [BEAM-12738] - Fix Dataflow cloud console URL
  • [BEAM-12759] - ORDER BY column not in SELECT crashes
  • [BEAM-12765] - Cannot access field fruit on a value with type ARRAY<STRUCT<fruit STRING>>
  • [BEAM-12768] - test_df_agg_method_invalid_kwarg_raises test too strict on error/string matching
  • [BEAM-12796] - Executing a BQ pipeline without the new BQ read API library results in a TypeError
  • [BEAM-12800] - Please triage performance regression: TFRecordIO 1GB GCS
  • [BEAM-12801] - Please triage performance regression: HadoopFormatIO
  • [BEAM-12805] - Multiple XVR suites failing due to Java test ValidateRunnerXlangTest.combinePerKeyTest
  • [BEAM-12829] - beam_Release_Gradle_Build failure: gofmt failure and copy failure
  • [BEAM-12838] - Update artifact local path for DataflowRunner Java
  • [BEAM-13119] - Add subdirectory prefixed tags to release process to support Go SDK versioning.

New Feature

  • [BEAM-12609] - Projection pushdown in SchemaIO
  • [BEAM-12665] - Add option to return filename from ReadAll transforms
  • [BEAM-12810] - Revert https://github.com/apache/beam/pull/15185 on 2.33.0 release branch

Improvement

  • [BEAM-3878] - Improve error reporting in calls.go
  • [BEAM-4152] - Support Go session windowing
  • [BEAM-5379] - Go Modules versioning support
  • [BEAM-7745] - StreamingSideInputDoFnRunner/StreamingSideInputFetcher have suboptimal state access pattern during normal operation
  • [BEAM-11853] - Build fails on macOS ARM64, because the supported version of go is 1.16.x or later
  • [BEAM-12428] - Implement TokenCredentialSerializer in AzureModule.java
  • [BEAM-12429] - Add support for S3 Bucket Key at the object level
  • [BEAM-12545] - WriteToFile should support mime/encoding.
  • [BEAM-12601] - Support append-only indices in ES output
  • [BEAM-12619] - Swap LinkedBlockingQueue to ArrayBlockingQueue
  • [BEAM-12640] - Add support for running JMH benchmarks
  • [BEAM-12704] - Fix primitive Read on portable Flink Runner
  • [BEAM-12735] - Adding Python XLang examples to the RC validation script
  • [BEAM-12739] - Flink JobServer should bundle AWS IO libraries to support staging on S3
  • [BEAM-12755] - Clean-up HistogramData to not throw errors during formatting
  • [BEAM-12778] - Prevent unnecessary dry run requests to BQ when temp dataset is provided by the user

Test

  • [BEAM-12710] - TestStream.testFirstElementLate is flaky on FlinkRunner by design

Task

Edit/Copy Release Notes

The text area below allows the project release notes to be edited and copied to another document.