Sub-task

• [BEAM-772] - Implement Metrics support for Dataflow Runner
• [BEAM-773] - Implement Metrics support for Flink runner
• [BEAM-775] - Remove Aggregators from the Java SDK
• [BEAM-827] - Remove PipelineOptions from construction time in WriteFiles
• [BEAM-1617] - Add Gauge metric type to Java SDK
• [BEAM-1651] - Add code style xml to the project repository
• [BEAM-1684] - Add unit tests for iobase.py
• [BEAM-1722] - Move PubsubIO out of the core SDK
• [BEAM-1726] - Verify PAssert execution in TestFlinkRunner
• [BEAM-1763] - TestPipeline should ensure that all assertions succeeded
• [BEAM-1912] - Move HashingFn into io/common so it can be used by other tests
• [BEAM-1958] - Standard IO Metrics in Java SDK
• [BEAM-2002] - Verify PAssert execution in TestSparkRunner
• [BEAM-2003] - Verify PAssert execution in TestDataflowRunner
• [BEAM-2030] - Implement beam FileSystem's copy()
• [BEAM-2031] - Hadoop FileSystem needs to receive Hadoop Configuration
• [BEAM-2032] - Implement delete
• [BEAM-2033] - Implement ResourceIds for HadoopFileSystem
• [BEAM-2070] - Implement match for HadoopFileSystem
• [BEAM-2329] - ABS Function
• [BEAM-2330] - MOD Function
• [BEAM-2331] - SQRT Function

Bug

• [BEAM-145] - OutputTimeFn#assignOutputTime overrides WindowFn#getOutputTime in unfortunate ways
• [BEAM-260] - WindowMappingFn: Know the getSideInputWindow upper bound to release side input resources
• [BEAM-437] - Data-dependent BigQueryIO in batch
• [BEAM-463] - BoundedHeapCoder should be a StandardCoder and not a CustomCoder
• [BEAM-539] - Error when writing to the root of a GCS location
• [BEAM-632] - Dataflow runner does not correctly flatten duplicate inputs
• [BEAM-655] - Rename @RunnableOnService to something more descriptive
• [BEAM-662] - SlidingWindows should support sub-second periods
• [BEAM-828] - Remove PipelineOptions from construction time in BigQueryIO
• [BEAM-1013] - Recheck all existing programming guide code snippets for correctness
• [BEAM-1022] - WindowNamespace and WindowAndTriggerNamespace should not use Java object equality when comparing windows
• [BEAM-1040] - Hadoop InputFormat - IO Transform for reads
• [BEAM-1048] - Spark Runner streaming batch duration does not include duration of reading from source
• [BEAM-1053] - ApexGroupByKeyOperator serialization issues
• [BEAM-1068] - Service Account Credentials File Specified via Pipeline Option Ignored
• [BEAM-1101] - Remove inconsistencies in Python PipelineOptions
• [BEAM-1213] - WordCount example failure on Apex Runner
• [BEAM-1247] - Session state should not be lost when discardingFiredPanes
• [BEAM-1264] - Python ChannelFactory Raise Inconsistent Error for Local FS and GCS
• [BEAM-1283] - DoFn finishBundle should be required to specify the window for output
• [BEAM-1316] - DoFn#startBundle should not be able to output
• [BEAM-1355] - HDFS IO should comply with PTransform style guide
• [BEAM-1362] - Update the beam release process to include python sdk
• [BEAM-1366] - Add metrics checks to Python SDK once metrics have been implemented
• [BEAM-1381] - Implement DataflowMetrics.query method
• [BEAM-1383] - Consistency in the Metrics examples
• [BEAM-1402] - Make TextIO and AvroIO use best-practice types.
• [BEAM-1414] - CountingInput should comply with PTransform style guide
• [BEAM-1415] - PubsubIO should comply with PTransform style guide
• [BEAM-1418] - MapElements and FlatMapElements should comply with PTransform style guide
• [BEAM-1422] - ParDo should comply with PTransform style guide
• [BEAM-1425] - Window should comply with PTransform style guide
• [BEAM-1428] - KinesisIO should comply with PTransform style guide
• [BEAM-1459] - Dataflow runner has deprecated metricsUpdates in favor of counterUpdates. Add setters.
• [BEAM-1508] - PInput, POutput#expand should not be ordered
• [BEAM-1546] - Specify exact version for Python in the SDK
• [BEAM-1568] - Ineffective null check in IsmFormat#structuralValue
• [BEAM-1569] - HDFSFileSource: Unable to read from filePattern with spaces in path
• [BEAM-1571] - Flatten on a single input PCollection should have a test associated with it
• [BEAM-1572] - Add per-stage matching of scope in metrics for the DirectRunner
• [BEAM-1575] - Add ValidatesRunner test to PipelineTest.test_metrics_in_source
• [BEAM-1578] - Runners should put PT overrides into a list rather than map
• [BEAM-1579] - Runners should verify that PT overrides converged
• [BEAM-1580] - Typo in bigquery_tornadoes example
• [BEAM-1594] - Treat JOB_STATE_DRAINED as terminal in DataflowRunner
• [BEAM-1629] - Metrics/aggregators accumulators should be instantiated before traversing pipeline
• [BEAM-1635] - TypeError in AfterWatermark class's __repr__ method
• [BEAM-1642] - Combine transformation evaluation fails on direct runner with Avro as a fallback coder
• [BEAM-1644] - IO ITs: shared directory for kubernetes resources and PipelineOptions?
• [BEAM-1645] - Display data not populated on Window.Assign
• [BEAM-1649] - Fix unresolved references in Python SDK
• [BEAM-1653] - Error when using PubsubIO with the DirectRunner
• [BEAM-1656] - DirectRunner should not call finalize twice in UnboundedSourceExecutorFactory
• [BEAM-1657] - DirectRunner should not call close twice in UnboundedSourceExecutorFactory
• [BEAM-1671] - Support bypassing `validate` flag when using tfrecordio
• [BEAM-1673] - PubSubIO can't write attributes
• [BEAM-1676] - SdkCoreApiSurfaceTest Failed When Directory Contains Space
• [BEAM-1686] - MQTT IO throws exception when client id is not specified
• [BEAM-1690] - BigQueryTornadoesIT failing
• [BEAM-1694] - Fix docstring inaccuracies in Python-SDK
• [BEAM-1695] - Improve Python-SDK's programming guide
• [BEAM-1709] - Implement Single-output ParDo as Multi-output ParDo
• [BEAM-1711] - Document extra features on quick start guide
• [BEAM-1713] - SparkRuntimeContext instances are leaking via StateSpecFunctions#mapSourceFunction
• [BEAM-1718] - Returning Duration.millis(Long.MAX_VALUE) in DoFn.getAllowedTimestampSkew() causes Overflow/Underflow
• [BEAM-1719] - Test modules are included in generated documentation
• [BEAM-1721] - Reshuffle can shift elements in time
• [BEAM-1723] - FlinkRunner should deduplicate when an UnboundedSource requires Deduping
• [BEAM-1732] - Window.Assign does not properly populate DisplayData of the enclosing Window transform
• [BEAM-1737] - Implement a Single-output ParDo as a Multi-output ParDo with a single output
• [BEAM-1741] - Update runner pages for Python
• [BEAM-1742] - UnboundedSource CheckpointMark should have more precise documentation
• [BEAM-1751] - Singleton ByteKeyRange with BigtableIO and Dataflow runner
• [BEAM-1762] - Python SDK Error Message no python 3 compatible
• [BEAM-1767] - Remove Aggregators from Dataflow runner
• [BEAM-1768] - assert_that always passes for empty inputs
• [BEAM-1769] - Travis - python only executes py27 tox environment
• [BEAM-1770] - DoFn javadoc claims no runner supports state or timers
• [BEAM-1772] - Support merging WindowFn other than IntervalWindow on Flink Runner
• [BEAM-1776] - Timers should be delivered in the window they were set in
• [BEAM-1777] - If PipelineEnforcement throws an exception after Pipeline.run() fails, it overwrites the original failure
• [BEAM-1780] - BigtableReader.splitIntoFraction should more carefully guard input
• [BEAM-1784] - DataflowPipelineJob.cancel() should be idempotent
• [BEAM-1792] - Spark runner uses its own filtering logic to match metrics
• [BEAM-1793] - Frequent python post commit errors
• [BEAM-1795] - Upgrade google-cloud-bigquery to 0.23.0
• [BEAM-1801] - default_job_name can generate names not accepted by DataFlow
• [BEAM-1802] - Spark Runner does not shutdown correctly when executing multiple pipelines in sequence
• [BEAM-1803] - Metrics filters have a missmatch in class-based namespace
• [BEAM-1810] - Spark runner combineGlobally uses Kryo serialization
• [BEAM-1815] - Avoid shuffling twice in GABW
• [BEAM-1818] - Expose side-channel inputs in PTransform
• [BEAM-1828] - GlobalWatermarkHolder uses unpersist instead of destory
• [BEAM-1832] - Potentially unclosed OutputStream in ApexYarnLauncher
• [BEAM-1835] - NPE in DirectRunner PubsubReader.ackBatch
• [BEAM-1837] - NPE in KafkaIO writer
• [BEAM-1838] - GlobalWindow equals() and hashCode() doesn't work with other serialization frameworks
• [BEAM-1842] - Stop matching composite PCollectionView PTransforms
• [BEAM-1844] - test_memory_usage fails in post commit
• [BEAM-1849] - Output from OnTimer method has windows re-assigned
• [BEAM-1856] - HDFSFileSink class do not use the same configuration in master and slave
• [BEAM-1862] - SplittableDoFnOperator should close the ScheduledExecutorService
• [BEAM-1865] - Input Coder of GroupByKey should be a KV Coder in the Python SDK
• [BEAM-1867] - Element counts missing on Cloud Dataflow when PCollection has anything other than hardcoded name pattern
• [BEAM-1869] - getProducingTransformInternal should not be available on any PValue
• [BEAM-1873] - Javadoc in BigQueryIO doesn't reflect recent changes
• [BEAM-1886] - Remove TextIO override in Flink runner
• [BEAM-1902] - Datastore IO never retries on errors
• [BEAM-1903] - Splittable DoFn should report watermarks via ProcessContext
• [BEAM-1904] - Remove DoFn.ProcessContinuation
• [BEAM-1913] - TFRecordIO should comply with PTransform style guide
• [BEAM-1914] - XML IO should comply with PTransform style guide
• [BEAM-1922] - DataSource in JdbcIO is not closed
• [BEAM-1926] - Need 3 Python snippets for composite transforms section in programming guide
• [BEAM-1935] - DirectRunner Cancel should never throw a RejectedExecutionException
• [BEAM-1937] - PipelineSurgery renumbers already-unique transforms
• [BEAM-1947] - DisplayData raises exception when passed unicode string
• [BEAM-1954] - "test" extra need nose in the requirements list
• [BEAM-1963] - Quick start on home page redirects to java quickstart
• [BEAM-1964] - Upgrade pylint to 1.7.0
• [BEAM-1966] - ApexRunner in cluster mode does not register standard FileSystems/IOChannelFactories
• [BEAM-1969] - GCP extras should not required fix version of proto-google-cloud-datastore-v1
• [BEAM-1970] - Cannot run UserScore on Flink runner due to AvroCoder classload issues
• [BEAM-1972] - HIFIO jdk module fails enforcer when only java 7 is installed on machine
• [BEAM-1977] - PubsubIO fails with NPE on ACK when running locally
• [BEAM-1981] - Serialization error with TimerInternals in ApexGroupByKeyOperator
• [BEAM-1988] - utils.path.join does not correctly handle GCS bucket roots
• [BEAM-1989] - clean SyntaxWarning
• [BEAM-1992] - Count.perElement javadoc refers to Count.PerElement, but Count.PerElement is private
• [BEAM-1998] - Update json_values_test.py for ValueProvider
• [BEAM-2017] - DataflowRunner: fix NullPointerException that can occur when no metrics are present
• [BEAM-2019] - Count.globally() requires default values for non-GlobalWindows
• [BEAM-2022] - ApexTimerInternals seems to treat processing time timers as event time timers
• [BEAM-2023] - BigQueryIO.Write needs a way of dynamically specifying table schemas
• [BEAM-2029] - NullPointerException when using multi output ParDo in Spark runner in streaming mode.
• [BEAM-2040] - Occasional build failures caused by AutoValue
• [BEAM-2052] - Windowed file sinks should support dynamic sharding
• [BEAM-2071] - AttributeError in dataflow_metrics
• [BEAM-2072] - MicrobatchSource.reader stops reading after reaching maxNumRecords for the first time
• [BEAM-2073] - Change SourceDStream.rateControlledMaxRecords() to better reflect its intention
• [BEAM-2074] - SourceDStream's rate control mechanism may not work
• [BEAM-2077] - Remove AvroCoder#createDatum(Reader/Writer)
• [BEAM-2084] - Distribution metrics should be queriable in the Dataflow Runner
• [BEAM-2086] - TestDataflowRunner relies on metrics which are not present in streaming jobs
• [BEAM-2091] - Typo in build instructions in Apex Runner's README.md
• [BEAM-2092] - MicrobatchSource can be relieved of some of its methods since it's never used as an actual BoundedSource
• [BEAM-2093] - Update Jackson version to 2.8.8 in archetype (or align with parent pom)
• [BEAM-2094] - WordCount examples produce garbage for non-English input text
• [BEAM-2095] - The hasNext method of the iterator returned by SourceRDD#compute is not idempotent
• [BEAM-2096] - NullPointerException in DataflowMetrics
• [BEAM-2098] - Walkthrough URL in example code Javadoc is 404 not found
• [BEAM-2105] - Audit that user-facing stuff is in main jars, not the test suite jars
• [BEAM-2106] - NotSerializableException thrown when serializing EvaluationContext
• [BEAM-2113] - Apex Runner is not able to submit any job to YARN
• [BEAM-2114] - KafkaIO broken with CoderException
• [BEAM-2116] - PubsubJsonClient doesn't write user created attributeMap
• [BEAM-2119] - FileSystems doesn't install the local filesystem on intialization by default
• [BEAM-2120] - DataflowPipelineJob processes all log messages with each waitUntilFinish
• [BEAM-2122] - Writing to partitioned BigQuery tables from Dataflow is causing errors
• [BEAM-2130] - Ensure options id is never null
• [BEAM-2136] - AvroCoderTest.testTwoClassLoaders fails on beam_PostCommit_Java_ValidatesRunner_Dataflow
• [BEAM-2143] - (Mis)Running Dataflow Wordcount gives non-helpful errors
• [BEAM-2152] - Authentication fails if there is an unauthenticated gcloud tool even if application default credentials are available
• [BEAM-2154] - Writing to large numbers of BigQuery tables causes out-of-memory
• [BEAM-2157] - HadoopFileSystemModuleTest Failed in Some JDK Versions on Jenkins
• [BEAM-2162] - Add logging during and after long running BigQuery jobs
• [BEAM-2170] - PubsubIO.readStrings should handle messages without metadata
• [BEAM-2181] - Upgrade Bigtable dependency to 0.9.6.2
• [BEAM-2183] - Maven-archetypes should depend on all Beam modules that their sources compile against
• [BEAM-2184] - OutputTimeFn is not a Fn: in Python, rename to TimestampCombiner
• [BEAM-2187] - SparkRuntimeContextTest fails to compile
• [BEAM-2190] - User depending on IO-GCP still gets a dependency on protobuf-lite
• [BEAM-2205] - AttributeError when running datastore wordcount
• [BEAM-2210] - PubsubIO.readPubsubMessagesWithoutAttributes is awkward
• [BEAM-2211] - DataflowRunner (Java) rejects all but GCS paths for FileBasedSource/Sink
• [BEAM-2212] - ValueProvider-ification of core transforms makes logs and errors worse
• [BEAM-2213] - Java DirectRunner takes 60s to shut down after wordcount runs
• [BEAM-2222] - Clean up readme files
• [BEAM-2223] - java8 examples are not running
• [BEAM-2224] - maptask_executor_runner_test fails in windows
• [BEAM-2229] - GcsFileSystem attempts to create invalid Metadata
• [BEAM-2233] - Java 8 examples should separate runners into distinct profiles, like Java 7 examples
• [BEAM-2236] - Move test utilities out of python core
• [BEAM-2239] - Step context not always available when exceptions raised.
• [BEAM-2240] - Step context not always available when exceptions raised.
• [BEAM-2242] - Apache Beam Java modules do not correctly shade test artifacts
• [BEAM-2243] - org.apache.beam.GcpCoreApiSurfaceTest.testApiSurface fails at release-2.0.0 head
• [BEAM-2244] - Move runner-facing Metrics classes to runners core
• [BEAM-2249] - AvroIO does not handle partial reads
• [BEAM-2256] - mongodb sdk MongoDbIO.BoundedMongoDbSource.splitKeysToFilters incorrect
• [BEAM-2259] - Reshuffle may set watermark holds past the end of time
• [BEAM-2260] - When using WindowedWrites and default FilenamePolicy, TextIO should throw at construction time
• [BEAM-2275] - SerializableCoder fails to serialize when used with a generic type token
• [BEAM-2277] - IllegalArgumentException when using Hadoop file system for WordCount example.
• [BEAM-2279] - Hadoop file system support should be included in examples/archetype profiles of Spark runner.
• [BEAM-2305] - Dinstinct transform produces unexpected output when triggered
• [BEAM-2326] - Verbose INFO logging with stateful DoFns and Dataflow
• [BEAM-2429] - Conflicting filesystems with used of HadoopFileSystem

New Feature

• [BEAM-59] - Switch from IOChannelFactory to FileSystems
• [BEAM-73] - IO design pattern: Decouple Parsers and Coders
• [BEAM-135] - Utilities for "batching" elements in a DoFn
• [BEAM-147] - Introduce an easy API for pipeline metrics
• [BEAM-404] - PubsubIO should have a mode that supports maintaining message attributes.
• [BEAM-596] - Support cancel() and waitUntilFinish() in DirectRunner
• [BEAM-638] - Add sink transform to write bounded data per window, pane, [and key] even when PCollection is unbounded
• [BEAM-846] - Decouple side input window mapping from WindowFn
• [BEAM-885] - Move PipelineOptions from Pipeline.create() to Pipeline.run()
• [BEAM-1047] - DataflowRunner: support regionalization.
• [BEAM-1076] - DatastoreIO template Options
• [BEAM-1195] - Give triggers a cross-language serialization schema
• [BEAM-1198] - ViewFn: explicitly decouple runner materialization of side inputs from SDK-specific mapping
• [BEAM-1327] - Replace OutputTimeFn with enum
• [BEAM-1328] - Serialize/deserialize WindowingStrategy in a language-agnostic manner
• [BEAM-1397] - Introduce IO metrics
• [BEAM-1398] - KafkaIO metrics
• [BEAM-1441] - Add FileSystem support to Python SDK
• [BEAM-1855] - Support Splittable DoFn in Flink Streaming runner
• [BEAM-1960] - Hadoop InputFormat - Add Kubernetes large and small cluster Scripts for Cassandra and Elasticsearch tests
• [BEAM-2005] - Add a Hadoop FileSystem implementation of Beam's FileSystem
• [BEAM-2054] - Upgrade dataflow.version to v1b3-rev196-1.22.0
• [BEAM-2147] - Re-enable UsesTimersInParDo tests for DataflowRunner

Improvement

• [BEAM-447] - Stop referring to types with Bound/Unbound
• [BEAM-649] - Smarter caching of RDDs
• [BEAM-720] - Run WindowedWordCount Integration Test in Flink
• [BEAM-806] - Maven Release Plugin Does Not Set Archetype Versions
• [BEAM-818] - ValueProvider for tempLocation, runner, etc, that is unavailable to transforms during construction
• [BEAM-831] - ParDo Chaining
• [BEAM-848] - Shuffle input read-values to get maximum parallelism.
• [BEAM-911] - Mark API of multiple IOs as @Experimental
• [BEAM-1071] - Support pre-existing tables with streaming BigQueryIO
• [BEAM-1074] - Set default-partitioner in SourceRDD.Unbounded.
• [BEAM-1148] - Port PAssert away from Aggregators
• [BEAM-1179] - Update assertions of source_test_utils from camelcase to underscore-separated
• [BEAM-1182] - Direct runner should enforce encodability of unbounded source checkpoints
• [BEAM-1199] - Condense recordAsOutput, finishSpecifyingOutput from POutput
• [BEAM-1242] - convert older IO/Sources to use standard ReadTransform style
• [BEAM-1269] - BigtableIO should make more efficient use of connections
• [BEAM-1272] - Align the naming of "generateInitialSplits" and "splitIntoBundles" to better reflect their intention
• [BEAM-1294] - Long running UnboundedSource Readers
• [BEAM-1336] - A StateSpec that doesn't care about the key shouldn't be forced to declare it as type Object
• [BEAM-1337] - Use our coder infrastructure for coders for state
• [BEAM-1340] - Remove or make private public bits of the SDK that shouldn't be public
• [BEAM-1345] - Mark @Experimental and @Internal where needed in user-facing bits of the codebase
• [BEAM-1401] - Sinks in Beam should supported windowed unbounded PCollections
• [BEAM-1447] - Autodetect streaming/not streaming in DataflowRunner
• [BEAM-1491] - HadoopFileSystemOptions should be able to read the HADOOP_CONF_DIR(YARN_CONF_DIR) environment variable
• [BEAM-1514] - change default timestamp in KafkaIO
• [BEAM-1520] - Implement TFRecordIO (Reading/writing Tensorflow Standard format)
• [BEAM-1530] - BigQueryIO should support value-dependent windows
• [BEAM-1539] - Support unknown length iterables for IterableCoder in Python SDK
• [BEAM-1562] - Use a "signal" to stop streaming tests as they finish.
• [BEAM-1573] - KafkaIO does not allow using Kafka serializers and deserializers
• [BEAM-1633] - Move .tox/ directory under target/ in Python SDK
• [BEAM-1660] - withCoder() error in JdbcIO JavaDoc example
• [BEAM-1661] - shade guava in beam-sdks-java-io-jdbc
• [BEAM-1672] - Accumulable MetricsContainers.
• [BEAM-1689] - Apply changes for Flink's StatefulDoFnRunner to the primary StatefulDoFnRunner
• [BEAM-1693] - Detect supported Python & pip executables in Python-SDK
• [BEAM-1704] - Create.TimestampedValues should take a TypeDescriptor as an alternative to explicitly specifying the Coder
• [BEAM-1708] - Better error messages when GCP features are not installed
• [BEAM-1727] - Add setForNowAlign(period, offset) to Timer
• [BEAM-1740] - Update bigtable version to 0.9.5.1
• [BEAM-1743] - View.AsSingleton should be implemented in terms of a Global Combine, not the reverse
• [BEAM-1749] - Upgrade pep8 to pycodestyle
• [BEAM-1786] - AutoService registration of coders, like we do with PipelineRunners
• [BEAM-1794] - Bigtable: improve user agent
• [BEAM-1799] - IO ITs: simplify data loading design pattern
• [BEAM-1807] - IO ITs: shared language neutral directory for kubernetes resources
• [BEAM-1812] - Allow configuring checkpoints in Flink Runner PipelineOptions
• [BEAM-1827] - Fix use of deprecated Spark APIs in the runner.
• [BEAM-1829] - MQTT message compression not working on Rapsberry Pi
• [BEAM-1830] - add 'withTopic()' api to KafkaIO Reader
• [BEAM-1839] - Optimize StatelessJavaSerializer
• [BEAM-1851] - Sample.fixedSizedGlobally documentation should include single worker memory constraint
• [BEAM-1858] - improve error message when Create.of() is called with an empty iterator
• [BEAM-1863] - Allow users to override the base container image but still choose image type
• [BEAM-1864] - Shorten combining state names: "CombiningValue" and "AccumulatorCombiningState" to Combining (as appropriate)
• [BEAM-1870] - ByteKey / ByteKeyRangeTracker should not use ByteString on public API surface
• [BEAM-1871] - Thin Java SDK Core
• [BEAM-1875] - Remove Spark runner custom Hadoop and Avro IOs.
• [BEAM-1876] - GroupIntoBatches may be able to use Combine.BinaryCombineLongFn
• [BEAM-1877] - Use Iterables.isEmpty in GroupIntoBatches
• [BEAM-1882] - Jdbc k8s scripts: switch pod -> replicaController
• [BEAM-1895] - Create tranform in python sdk should be a custom source
• [BEAM-1897] - Remove Sink
• [BEAM-1907] - Delete PubsubBoundedReader
• [BEAM-1908] - Allow setting CREATE_NEVER when using a tablespec in BigQueryIO
• [BEAM-1921] - expose connectionProperties in JdbcIO
• [BEAM-1923] - Improve python log messages for temporary BigQuery tables
• [BEAM-1949] - Rename DoFn.Context#sideOutput to #output
• [BEAM-1990] - Window.Assign should not be public since it is not meant to be used publicly
• [BEAM-1991] - Update references to SumDoubleFn => Sum.ofDoubles
• [BEAM-1993] - Remove special unbounded Flink source/sink
• [BEAM-1994] - Remove Flink examples package
• [BEAM-2013] - Upgrade to Jackson 2.8.8
• [BEAM-2014] - Upgrade to Google Auth 0.6.1
• [BEAM-2020] - Move CloudObject to Dataflow runner
• [BEAM-2021] - Fix Java's Coder class hierarchy
• [BEAM-2044] - Downgrade HBaseIO to use the stable HBase client version (1.2.x)
• [BEAM-2047] - PubsubStreamingWrite should use the input coder by default
• [BEAM-2049] - Remove KeyedCombineFn
• [BEAM-2051] - Reduce scope of the PCollectionView interface
• [BEAM-2060] - XmlIO use harcoded Charset
• [BEAM-2062] - EventHandler jaxb unmarshaller should be optional
• [BEAM-2067] - Add support for generic CoderProvider -> CoderFactory mapping with CoderRegistrar
• [BEAM-2068] - Upgrade Google-Apitools to latest version
• [BEAM-2075] - Update flink runner to use flink version 1.2.1
• [BEAM-2076] - DirectRunner: minimal transitive API surface
• [BEAM-2099] - Create a WordCount example that works with HDFS
• [BEAM-2135] - Rename hdfs module to hadoop-file-system, rename gcp-core to google-cloud-platform-core
• [BEAM-2144] - Do not publish javadoc for Java SDK's util directory
• [BEAM-2165] - Support custom user Jackson modules for PipelineOptions
• [BEAM-2166] - Remove Coder.Context from the public API
• [BEAM-2174] - Allow coder factories to create Coders for a wider range of types
• [BEAM-2206] - Move pipeline options into separate package from beam/utils
• [BEAM-2218] - PubsubIO.readPubsubMessages function names are too long
• [BEAM-2221] - Make KafkaIO coder specification less awkward
• [BEAM-2241] - Correctly mark top level classes and functions as private
• [BEAM-2245] - Remove user-facing Timer.cancel() until further notice
• [BEAM-2250] - Remove FnHarness code from PyDocs
• [BEAM-3770] - The problem of kafkaIO sdk for data latency

Test

• [BEAM-1184] - Add integration tests for ElasticsearchIO
• [BEAM-1622] - Java: Rename RunnableOnService to ValidatesRunner
• [BEAM-1752] - Tag Spark runner tests that recover from checkpoint.
• [BEAM-2057] - Test metrics are reported to Spark Metrics sink.
• [BEAM-2368] - one throw "Unable to find registrar for hdfs" with same code
• [BEAM-3383] - Create validates runner metrics tests

Wish

• [BEAM-378] - Integrate Python SDK in the Maven build
• [BEAM-797] - A PipelineVisitor that creates a Spark-native pipeline.
• [BEAM-1648] - Replace gsutil calls with Cloud Storage API

Task

• [BEAM-825] - Fill in the documentation/runners/apex portion of the website
• [BEAM-1027] - Hosting data stores to enable IO Transform testing
• [BEAM-1353] - Beam should comply with PTransform style guide
• [BEAM-1764] - Remove Aggregators from Flink Runner
• [BEAM-1765] - Remove Aggregators from Spark runner
• [BEAM-1766] - Remove Aggregators from Apex runner
• [BEAM-1797] - add CoGroupByKey to chapter 'Using GroupByKey'
• [BEAM-1887] - Switch ParDo execution to use new DoFn in Apex runner
• [BEAM-1915] - Remove OldDoFn dependency in ApexGroupByKeyOperator
• [BEAM-2016] - Delete HDFSFileSource/Sink
• [BEAM-2124] - Deprecate <pipeline>.options usage
• [BEAM-2139] - Disable SplittableDoFn ValidatesRunner tests for Streaming Flink Runner
• [BEAM-2180] - Upgrade Apex dependency to 3.6.0
• [BEAM-2235] - Restore wordcount example to its previous state

Edit/Copy Release Notes

The text area below allows the project release notes to be edited and copied to another document.