Sub-task
- [BEAM-10212] - Add support for Java SDK harness state caching
- [BEAM-13507] - Python PostCommit Failures -
- [BEAM-13508] - Java PostCommits (Examples, PerformanceTests, XVR] Failing with ClassNotFound
- [BEAM-13645] - Vendor gRPC 1.43.2
- [BEAM-13646] - Vendor calcite with protobuf 3.19.2
Bug
- [BEAM-12587] - Python "object" row type rejects null values.
- [BEAM-12869] - Upgrade tensorflow to 2.6.x
- [BEAM-13009] - DynamoDBIO misses writing items if `withDeduplicateKeys` is not set
- [BEAM-13209] - DynamoDBIO silently drops unprocessed items
- [BEAM-13213] - OnWindowExpiration does not work without other state
- [BEAM-13286] - RowJson fails to serialize DATETIMEs produced by ZetaSQL
- [BEAM-13294] - innerBroadcastJoin fails when batch column Nullable, streaming column not
- [BEAM-13313] - Use snake case for field name for WindowingStrategy proto
- [BEAM-13341] - Serialization of AvroCoder for dataflow workers discard reflect option
- [BEAM-13354] - Java SDK Harness MultimapUserState not using structural value when comparing keys
- [BEAM-13374] - ToJson() of a Row with a DateTime field fails with an exception
- [BEAM-13402] - Pub/Sub Lite commits can take 10s of hours without completing
- [BEAM-13421] - Python DeferredDataFrame.xs differs from Pandas
- [BEAM-13443] - Poor handling of aggregated records in KinesisIO.read
- [BEAM-13445] - Fix chunked encoding for S3WritableButeChannel
- [BEAM-13454] - Dataframe read_fwf fails reading incrementally.
- [BEAM-13456] - beam_PostCommit_Java consistently timing out.
- [BEAM-13467] - Schema translation to proto doesn't handle null arguements
- [BEAM-13526] - Unable to use the KafkaIO.Read.withValueDeserializer(DeserializerProvider) method
- [BEAM-13539] - Gradle warning: processResources overrides
- [BEAM-13599] - Overflow in Python Datastore RampupThrottlingFn
- [BEAM-13603] - Shared object does not read from cache when using tag
- [BEAM-13604] - NPE while getting null from BigDecimal column
- [BEAM-13611] - CrossLanguageJdbcIOTest.test_xlang_jdbc_write failing in Python PostCommits
- [BEAM-13616] - Update protobuf-java to 3.19.2 and other vendored dependencies that use protobuf
- [BEAM-13628] - [Go SDK] Make Side input cache fit resolved semantics.
- [BEAM-13665] - Spanner IO request metrics requires projectId within the config when it didn't in the past
- [BEAM-13668] - Java Spanner IO Request Count metrics broke backwards compatibility
- [BEAM-13716] - Clear before creating a new virtual environment in setupVirtualenv
- [BEAM-13830] - XVR Direct/Spark/Flink tests are timing out
- [BEAM-14004] - [CosmosDB] com.mongodb.MongoCommandException: Command failed with error 115: Command is not supported
New Feature
- [BEAM-12865] - Allow customising batch duration when streaming with WriteToBigQuery
- [BEAM-13052] - Pub/Sub Lite support for Python SDK
- [BEAM-13159] - Support for Redis 5.0 streams (XADD/XREAD/XRANGE) in RedisIO
- [BEAM-13351] - Add a connector for FhirIO's GetPatientEverything functionality
- [BEAM-13389] - Enable JdbcIO expansion service to more easily receive JDBC Driver as a dependency
- [BEAM-13617] - StorageApi sink might try to reuse closed stream in batch mode and in STORAGE_API_AT_LEAST_ONCE mode
- [BEAM-13697] - Add SchemaFieldNumber annotation
Improvement
- [BEAM-3304] - Go triggering support
- [BEAM-7559] - Add an Integration Test for DynamoDBIO
- [BEAM-9706] - Add ignoreUnknownValues flag to beam.io.WriteToBigQuery
- [BEAM-10584] - Replace getErrorInfo by quotaExceeded method in InsertAll for BigQuery
- [BEAM-11703] - Support apache-beam python install on ARM64
- [BEAM-12561] - Implement truncate for DataFrame and Series
- [BEAM-12565] - Implement compare for DataFrame and Series
- [BEAM-13171] - Support for stopReadTime on KafkaIO SDF
- [BEAM-13233] - Replace AWS API used to list Kinesis shards from DescribeStream to ListShards (SDK v2)
- [BEAM-13244] - Add STS Assume role credentials provider to AwsModule (AWS SDK v2)
- [BEAM-13276] - Beam Dependency Update Request: com.fasterxml.jackson.core
- [BEAM-13335] - DataFrame sources produce excessively large index
- [BEAM-13397] - Update numpy required version to >=1.21.0 for M1 chip
- [BEAM-13410] - Add integration test for S3FileSystem
- [BEAM-13430] - Upgrade Gradle version to 7.3
- [BEAM-13441] - Use quiet mode for batch deletes in AWS S3FileSystem
- [BEAM-13459] - Dataflow python runner should cache uploaded artifacts across job runs
- [BEAM-13481] - Upgrade shadow plugin (log4j)
- [BEAM-13488] - ClassGraph scanner is nondeterministic/incomplete when using parallel scans
- [BEAM-13494] - KinesisIO.read requires default rater limiter for low throughput shards
- [BEAM-13541] - Use runtime information to improve CoGroupByKey caching
- [BEAM-13591] - Bump log4j2 version to 2.17.1 due to the vulnerability
Task
- [BEAM-12733] - [beam_PostCommit_Java] org.apache.beam.sdk.extensions.ml.RecommendationAIPredictIT.predict failing
- [BEAM-13236] - Properly close kinesis producer on teardown
- [BEAM-13781] - grpc-netty-shaded version conflict
Edit/Copy Release Notes
The text area below allows the project release notes to be edited and copied to another document.