Release 1.0.0 – Changelog and Known Issues

Changelog

Sub-task

  • [FLINK-7] - [GitHub] Enable Range Partitioner
  • [FLINK-146] - Sorted output not working
  • [FLINK-1982] - Remove dependencies on Record for Flink runtime and core
  • [FLINK-2676] - Add abstraction for keyed window state
  • [FLINK-2680] - Create a dedicated aligned-event time window operator
  • [FLINK-2730] - Add CPU/Network utilization graphs to new web dashboard
  • [FLINK-2732] - Add access to the TaskManagers' log file and out file in the web dashboard.
  • [FLINK-2850] - Limit the types of jobs which can run in detached mode
  • [FLINK-2853] - Apply JMH on MutableHashTablePerformanceBenchmark class.
  • [FLINK-2869] - Apply JMH on IOManagerPerformanceBenchmark class.
  • [FLINK-2889] - Apply JMH on LongSerializationSpeedBenchmark class
  • [FLINK-2890] - Apply JMH on StringSerializationSpeedBenchmark class.
  • [FLINK-2900] - Remove Record-API dependencies from Hadoop Compat module
  • [FLINK-2901] - Several flink-test ITCases depend on Record API features
  • [FLINK-2906] - Remove Record-API
  • [FLINK-2919] - Apply JMH on FieldAccessMinibenchmark class.
  • [FLINK-2920] - Apply JMH on KryoVersusAvroMinibenchmark class.
  • [FLINK-2933] - Flink scala libraries exposed with maven should carry scala version
  • [FLINK-2972] - Remove Twitter Chill dependency from flink-java module
  • [FLINK-3057] - [py] Provide a way to pass information back to the plan process
  • [FLINK-3070] - Create an asynchronous state handle interface
  • [FLINK-3071] - Add asynchronous materialization thread
  • [FLINK-3140] - NULL value data layout in Row Serializer/Comparator
  • [FLINK-3195] - Restructure examples projects and package streaming examples
  • [FLINK-3201] - Enhance Partitioned State Interface with State Types
  • [FLINK-3208] - Rename Gelly vertex-centric model to scatter-gather
  • [FLINK-3224] - The Streaming API does not call setInputType if a format implements InputTypeConfigurable
  • [FLINK-3275] - [py] Add support for Dataset.setParallelism()
  • [FLINK-3285] - Skip Maven deployment of flink-java8
  • [FLINK-3312] - Add convenience accessor methods for extended state interface
  • [FLINK-3327] - Attach the ExecutionConfig to the JobGraph and make it accessible to the AbstractInvocable.
  • [FLINK-3363] - JobManager does not shut down dedicated executor properly
  • [FLINK-3365] - BlobLibraryCacheManager does not shutdown Timer thread
  • [FLINK-3371] - Move TriggerCotext and TriggerResult to their own classes
  • [FLINK-3384] - Create atomic closable queue for communication between Kafka Threads
  • [FLINK-3388] - Expose task accumulators via JMX
  • [FLINK-3401] - AscendingTimestampExtractor should not fail on order violation
  • [FLINK-3437] - Fix UI router state for job plan
  • [FLINK-3468] - Change Timestamp Extractors to not support negative timestamps
  • [FLINK-3470] - EventTime WindowAssigners should error on negative timestamps

Bug

  • [FLINK-1278] - Remove the Record special code paths
  • [FLINK-1644] - WebClient dies when no ExecutionEnvironment in main method
  • [FLINK-1989] - Sorting of POJO data set from TableEnv yields NotSerializableException
  • [FLINK-2115] - TableAPI throws ExpressionException for "Dangling GroupBy operation"
  • [FLINK-2263] - ExecutionGraph uses Thread.sleep to delay execution retries
  • [FLINK-2315] - Hadoop Writables cannot exploit implementing NormalizableKey
  • [FLINK-2348] - IterateExampleITCase failing
  • [FLINK-2351] - Deprecate config builders in InputFormats and Output formats
  • [FLINK-2369] - On Windows, in testFailingSortingDataSinkTask the temp file is not removed
  • [FLINK-2392] - Instable test in flink-yarn-tests
  • [FLINK-2443] - [CompactingHashTable] GSA Connected Components fails with NPE
  • [FLINK-2491] - Operators are not participating in state checkpointing in some cases
  • [FLINK-2504] - ExternalSortLargeRecordsITCase.testSortWithLongAndShortRecordsMixed failed spuriously
  • [FLINK-2586] - Unstable Storm Compatibility Tests
  • [FLINK-2608] - Arrays.asList(..) does not work with CollectionInputFormat
  • [FLINK-2622] - Scala DataStream API does not have writeAsText method which supports WriteMode
  • [FLINK-2624] - RabbitMQ source / sink should participate in checkpointing
  • [FLINK-2662] - CompilerException: "Bug: Plan generation for Unions picked a ship strategy between binary plan operators."
  • [FLINK-2671] - Instable Test StreamCheckpointNotifierITCase
  • [FLINK-2674] - Rework windowing logic
  • [FLINK-2695] - KafkaITCase.testConcurrentProducerConsumerTopology failed on Travis
  • [FLINK-2719] - ProcessFailureStreamingRecoveryITCase>AbstractProcessFailureRecoveryTest.testTaskManagerProcessFailure failed on Travis
  • [FLINK-2739] - Release script depends on the order of parent module information
  • [FLINK-2747] - TypeExtractor does not correctly analyze Scala Immutables (AnyVal)
  • [FLINK-2752] - Documentation is not easily differentiable from the Flink homepage
  • [FLINK-2757] - DataSinkTaskTest fails on Windows
  • [FLINK-2758] - TaskManagerRegistrationTest fails on Windows
  • [FLINK-2759] - TaskManagerProcessReapingTest fails on Windows
  • [FLINK-2769] - Web dashboard port not configurable on client side
  • [FLINK-2771] - IterateTest.testSimpleIteration fails on Travis
  • [FLINK-2797] - CLI: Missing option to submit jobs in detached mode
  • [FLINK-2799] - Yarn tests cannot be executed with DEBUG log level
  • [FLINK-2800] - kryo serialization problem
  • [FLINK-2823] - YARN client should report a proper exception if Hadoop Env variables are not set
  • [FLINK-2826] - transformed is modified in BroadcastVariableMaterialization#decrementReferenceInternal without proper locking
  • [FLINK-2827] - Potential resource leak in TwitterSource#loadAuthenticationProperties()
  • [FLINK-2832] - Failing test: RandomSamplerTest.testReservoirSamplerWithReplacement
  • [FLINK-2838] - Inconsistent use of URL and Path classes for resources
  • [FLINK-2845] - TimestampITCase.testWatermarkPropagation
  • [FLINK-2879] - Links in documentation are broken
  • [FLINK-2913] - Close of ObjectOutputStream should be enclosed in finally block in FsStateBackend
  • [FLINK-2914] - Missing break in ZooKeeperSubmittedJobGraphStore#SubmittedJobGraphsPathCacheListener#childEvent()
  • [FLINK-2930] - ExecutionConfig execution retry delay not respected
  • [FLINK-2934] - Remove placehoder pages in Web dashboard
  • [FLINK-2937] - Typo in Quickstart->Scala API->Alternative Build Tools: SBT
  • [FLINK-2938] - Streaming docs not in sync with latest state changes
  • [FLINK-2942] - Dangling operators in web UI's program visualization (non-deterministic)
  • [FLINK-2950] - Markdown presentation problem in SVM documentation
  • [FLINK-2954] - Not able to pass custom environment variables in cluster to processes that spawning TaskManager
  • [FLINK-2958] - StreamingJobGraphGenerator sets hard coded number execution retry
  • [FLINK-2963] - Dependence on SerializationUtils#deserialize() should be avoided
  • [FLINK-2967] - TM address detection might not always detect the right interface on slow networks / overloaded JMs
  • [FLINK-2977] - Cannot access HBase in a Kerberos secured Yarn cluster
  • [FLINK-2979] - RollingSink does not work with Hadoop 2.7.1
  • [FLINK-2987] - Flink 0.10 fails to start on YARN 2.6.0
  • [FLINK-2989] - Job Cancel button doesn't work on Yarn
  • [FLINK-2990] - Scala 2.11 build fails to start on YARN
  • [FLINK-2992] - New Windowing code is using SerializationUtils with wrong classloader
  • [FLINK-3000] - Add ShutdownHook to YARN CLI to prevent lingering sessions
  • [FLINK-3005] - Commons-collections object deserialization remote command execution vulnerability
  • [FLINK-3009] - Cannot build docs with Jekyll 3.0.0
  • [FLINK-3011] - Cannot cancel failing/restarting streaming job from the command line
  • [FLINK-3013] - Incorrect package declaration in GellyScalaAPICompletenessTest.scala
  • [FLINK-3019] - CLI does not list running/restarting jobs
  • [FLINK-3020] - Local streaming execution: set number of task manager slots to the maximum parallelism
  • [FLINK-3022] - Broken link 'Working With State' in Fault Tolerance Section of Stream Programming Guide
  • [FLINK-3024] - TimestampExtractor Does not Work When returning Long.MIN_VALUE
  • [FLINK-3025] - Flink Kafka consumer may get stuck due to Kafka/Zookeeper client bug
  • [FLINK-3032] - Flink does not start on Hadoop 2.7.1 (HDP), due to class conflict
  • [FLINK-3043] - Kafka Connector description in Streaming API guide is wrong/outdated
  • [FLINK-3047] - Local batch execution: set number of task manager slots to the maximum parallelism
  • [FLINK-3048] - DataSinkTaskTest.testCancelDataSinkTask
  • [FLINK-3052] - Optimizer does not push properties out of bulk iterations
  • [FLINK-3054] - Remove R (return) type variable from SerializationSchema
  • [FLINK-3059] - Javadoc fix for DataSet.writeAsText()
  • [FLINK-3061] - Kafka Consumer is not failing if broker is not available
  • [FLINK-3062] - Kafka Producer is not failing if broker is not available/no partitions available
  • [FLINK-3067] - Kafka source fails during checkpoint notifications with NPE
  • [FLINK-3080] - Cannot union a data stream with a product of itself
  • [FLINK-3081] - Kafka Periodic Offset Committer does not properly terminate on canceling
  • [FLINK-3082] - Confusing error about ManualTimestampSourceFunction
  • [FLINK-3087] - Table API do not support multi count in aggregation.
  • [FLINK-3098] - Cast from Date to Long throw compile error.
  • [FLINK-3100] - Signal handler prints error on normal shutdown of cluster
  • [FLINK-3101] - Flink Kafka consumer crashes with NPE when it sees deleted record
  • [FLINK-3103] - Remove synchronization in FsStateBackend#FsCheckpointStateOutputStream#close()
  • [FLINK-3108] - JoinOperator's with() calls the wrong TypeExtractor method
  • [FLINK-3117] - Storm Tick Tuples are not supported
  • [FLINK-3118] - Check if MessageFunction implements ResultTypeQueryable
  • [FLINK-3121] - Watermark forwarding does not work for sources not producing any data
  • [FLINK-3125] - Web dashboard does not start when log files are not found
  • [FLINK-3134] - Make YarnJobManager's allocate call asynchronous
  • [FLINK-3136] - Scala Closure Cleaner uses wrong ASM import
  • [FLINK-3138] - Method References are not supported as lambda expressions
  • [FLINK-3143] - Update Clojure Cleaner's ASM references to ASM5
  • [FLINK-3144] - [storm] LocalCluster prints nothing without a configured logger
  • [FLINK-3145] - Storm examples can't be run without flink-java as dependency
  • [FLINK-3151] - YARN kills Flink TM containers due to memory overuse (outside heap/offheap)
  • [FLINK-3156] - FlinkKafkaConsumer fails with NPE on notifyCheckpointComplete
  • [FLINK-3157] - Web frontend json files contain author attribution
  • [FLINK-3166] - The first program in ObjectReuseITCase has the wrong expected result, and it succeeds
  • [FLINK-3171] - Consolidate zoo of wrapper classes for input/output-stream to data-input/output-view
  • [FLINK-3173] - Bump org.apache.httpcomponents.httpclient version to 4.2.6
  • [FLINK-3175] - KafkaITCase.testOffsetAutocommitTest
  • [FLINK-3179] - Combiner is not injected if Reduce or GroupReduce input is explicitly partitioned
  • [FLINK-3180] - MemoryLogger does not log direct memory
  • [FLINK-3185] - Silent failure during job graph recovery
  • [FLINK-3188] - Deletes in Kafka source should be passed on to KeyedDeserializationSchema
  • [FLINK-3189] - Error while parsing job arguments passed by CLI
  • [FLINK-3196] - InputStream should be closed in EnvironmentInformation#getRevisionInformation()
  • [FLINK-3197] - InputStream not closed in BinaryInputFormat#createStatistics
  • [FLINK-3206] - Heap size for non-pre-allocated off-heap memory
  • [FLINK-3218] - Merging Hadoop configurations overrides user parameters
  • [FLINK-3220] - Flink does not start on Hortonworks Sandbox 2.3.2 due to missing class
  • [FLINK-3236] - Flink user code classloader should have Flink classloader as parent classloader
  • [FLINK-3240] - Remove or document DataStream(.global|.forward)
  • [FLINK-3242] - User-specified StateBackend is not Respected if Checkpointing is Disabled
  • [FLINK-3243] - Fix Interplay of TimeCharacteristic and Time Windows
  • [FLINK-3247] - Kafka Connector unusable with quickstarts - shading issue
  • [FLINK-3248] - RMQSource does not provide a constructor for credentials or other options
  • [FLINK-3250] - Savepoint coordinator requires too strict parallelism match
  • [FLINK-3251] - Checkpoint stats show ghost numbers
  • [FLINK-3252] - Checkpoint stats only displayed after click on graph
  • [FLINK-3254] - CombineFunction interface not respected
  • [FLINK-3255] - Chaining behavior should not depend on parallelism
  • [FLINK-3260] - ExecutionGraph gets stuck in state FAILING
  • [FLINK-3261] - Tasks should eagerly report back when they cannot start a checkpoint
  • [FLINK-3266] - LocalFlinkMiniCluster leaks resources when multiple jobs are submitted
  • [FLINK-3267] - Disable reference tracking in Kryo fallback serializer
  • [FLINK-3268] - Unstable test JobManagerSubmittedJobGraphsRecoveryITCase
  • [FLINK-3271] - Using webhdfs in a flink topology throws classnotfound exception
  • [FLINK-3274] - Prefix Kafka connector accumulators with unique id
  • [FLINK-3280] - Wrong usage of Boolean.getBoolean()
  • [FLINK-3281] - IndexOutOfBoundsException when range-partitioning empty DataSet
  • [FLINK-3286] - Remove JDEB Debian Package code from flink-dist
  • [FLINK-3287] - Flink Kafka Consumer fails due to Curator version conflict
  • [FLINK-3289] - Double reference to flink-contrib
  • [FLINK-3292] - Bug in flink-jdbc. Not all JDBC drivers supported
  • [FLINK-3293] - Custom Application Name on YARN is ignored in deploy jobmanager mode
  • [FLINK-3296] - DataStream.write*() methods are not flushing properly
  • [FLINK-3300] - Concurrency Bug in Yarn JobManager
  • [FLINK-3304] - AvroOutputFormat.setSchema() doesn't work in yarn-cluster mode
  • [FLINK-3314] - Early cancel calls can cause Tasks to not cancel properly
  • [FLINK-3322] - MemoryManager creates too much GC pressure with iterative jobs
  • [FLINK-3328] - Incorrectly shaded dependencies in flink-runtime
  • [FLINK-3333] - Documentation about object reuse should be improved
  • [FLINK-3337] - mvn test fails on flink-runtime because curator classes not found
  • [FLINK-3338] - Kafka deserialization issue - ClassNotFoundException
  • [FLINK-3339] - Checkpointing NPE when using filterWithState
  • [FLINK-3340] - Fix object juggling in drivers
  • [FLINK-3341] - Kafka connector's 'auto.offset.reset' inconsistent with Kafka
  • [FLINK-3342] - Operator checkpoint statistics state size overflow
  • [FLINK-3350] - Increase timeouts on Travis Builds
  • [FLINK-3351] - RocksDB Backend cannot determine correct local db path
  • [FLINK-3352] - RocksDB Backend cannot determine correct hdfs path
  • [FLINK-3353] - CSV-related tests may fail depending on locale
  • [FLINK-3357] - Drop JobId.toShortString()
  • [FLINK-3359] - Make RocksDB file copies asynchronous
  • [FLINK-3361] - Wrong error messages for execution retry delay and akka ask pause config values
  • [FLINK-3368] - Kafka 0.8 consumer fails to recover from broker shutdowns
  • [FLINK-3369] - RemoteTransportException should be instance of CancelTaskException
  • [FLINK-3373] - Using a newer library of Apache HttpClient than 4.2.6 will get class loading problems
  • [FLINK-3385] - Fix outer join skipping unprobed partitions
  • [FLINK-3392] - Unprotected access to elements in ClosableBlockingQueue#size()
  • [FLINK-3393] - ExternalProcessRunner wait to finish copying error stream
  • [FLINK-3394] - Clear up the contract of MutableObjectIterator.next(reuse)
  • [FLINK-3396] - Job submission Savepoint restore logic flawed
  • [FLINK-3400] - RocksDB Backend does not work when not in Flink lib folder
  • [FLINK-3412] - Remove implicit conversions JavaStream / ScalaStream
  • [FLINK-3415] - TimestampExctractor accepts negative watermarks
  • [FLINK-3416] - [py] .bat files fail when path contains spaces
  • [FLINK-3420] - Remove "ReadTextFileWithValue" from StreamExecutionEnvironment
  • [FLINK-3423] - ExternalProcessRunnerTest fails on Windows
  • [FLINK-3424] - FileStateBackendtest.testStateOutputStream fails on windows
  • [FLINK-3425] - DataSinkTaskTest.Failing[Sorting]DataSinkTask fails on Windows
  • [FLINK-3426] - JobManagerLeader[Re]ElectionTest.testleaderElection fails on Windows
  • [FLINK-3432] - ZookeeperOffsetHandlerTest fails on windows
  • [FLINK-3434] - Return value from flinkYarnClient#deploy() should be checked against null
  • [FLINK-3438] - ExternalProcessRunner fails to detect ClassNotFound exception because of locale settings
  • [FLINK-3439] - Remove final Long.MAX_VALUE Watermark in StreamSource
  • [FLINK-3440] - Kafka should also checkpoint partitions where no initial offset was retrieved
  • [FLINK-3448] - WebRuntimeMonitor: Move initialization of handlers to start() method
  • [FLINK-3449] - createInput swallows exception if TypeExtractor fails
  • [FLINK-3450] - RocksDB Backed Window Fails with KryoSerializer
  • [FLINK-3453] - Fix TaskManager logs exception when sampling backpressure while task completes
  • [FLINK-3458] - Shading broken in flink-shaded-hadoop
  • [FLINK-3478] - Flink serves arbitary files through the web interface
  • [FLINK-3483] - Job graph visualization not working properly in OS X Chrome
  • [FLINK-3485] - The SerializedListAccumulator value doest seem to be right
  • [FLINK-3499] - Possible ghost references in ZooKeeper completed checkpoint store
  • [FLINK-3511] - Flink library examples not runnable without adding dependencies
  • [FLINK-3512] - Savepoint backend should not revert to "jobmanager"
  • [FLINK-3513] - Fix interplay of automatic Operator UID and Changing name of WindowOperator
  • [FLINK-3518] - Stale docs for quickstart setup
  • [FLINK-3525] - Missing call to super#close() in TimestampsAndPeriodicWatermarksOperator#close()
  • [FLINK-3526] - ProcessingTime Window Assigner and Trigger are broken
  • [FLINK-3534] - Cancelling a running job can lead to restart instead of stopping
  • [FLINK-3540] - Hadoop 2.6.3 build contains /com/google/common (guava) classes in flink-dist.jar
  • [FLINK-3554] - Bounded sources should emit a Max Watermark when they are done
  • [FLINK-3556] - Unneeded check in HA blob store configuration
  • [FLINK-3562] - Update docs in the course of EventTimeSourceFunction removal
  • [FLINK-3565] - FlinkKafkaConsumer does not work with Scala 2.11
  • [FLINK-3569] - Test cases fail due to Maven Shade plugin

Improvement

  • [FLINK-8] - [GitHub] Implement automatic sample histogram building for Range Partitioning
  • [FLINK-553] - Add getGroupKey() method to group-at-time operators
  • [FLINK-573] - Clean-up MapOperators in optimizer
  • [FLINK-734] - Integrate web job client into JobManager web interface
  • [FLINK-987] - Extend TypeSerializers and -Comparators to work directly on Memory Segments
  • [FLINK-1045] - Remove Combinable Annotation
  • [FLINK-1228] - Add REST Interface to JobManager
  • [FLINK-1240] - We cannot use sortGroup on a global reduce
  • [FLINK-1513] - Remove GlobalConfiguration Singleton
  • [FLINK-1666] - Clean-up Field Expression Code
  • [FLINK-1702] - Authenticate via Kerberos from the client only
  • [FLINK-1778] - Improve normalized keys in composite key case
  • [FLINK-1903] - Joins where one side uses a field more than once don't work
  • [FLINK-1947] - Make Avro and Tachyon test logging less verbose
  • [FLINK-2017] - Add predefined required parameters to ParameterTool
  • [FLINK-2021] - Rework examples to use ParameterTool
  • [FLINK-2075] - Shade akka and protobuf dependencies away
  • [FLINK-2185] - Rework semantics for .setSeed function of SVM
  • [FLINK-2213] - Configure number of vcores
  • [FLINK-2239] - print() on DataSet: stream results and print incrementally
  • [FLINK-2342] - Add new fit operation and more tests for StandardScaler
  • [FLINK-2344] - Deprecate/Remove the old Pact Pair type
  • [FLINK-2380] - Allow to configure default FS for file inputs
  • [FLINK-2399] - Fail when actor versions don't match
  • [FLINK-2455] - Misleading I/O manager error log messages
  • [FLINK-2488] - Expose attemptNumber in RuntimeContext
  • [FLINK-2518] - Avoid predetermination of ports for network services
  • [FLINK-2523] - Make task canceling interrupt interval configurable
  • [FLINK-2524] - Add "getTaskNameWithSubtasks()" to RuntimeContext
  • [FLINK-2559] - Fix Javadoc (e.g. Code Examples)
  • [FLINK-2646] - Rich functions should provide a method "closeAfterFailure()"
  • [FLINK-2667] - Rework configuration parameters
  • [FLINK-2716] - Checksum method for DataSet and Graph
  • [FLINK-2788] - Add type hint with TypeExtactor call on Hint Type
  • [FLINK-2795] - Print JobExecutionResult for interactively invoked jobs
  • [FLINK-2882] - Improve performance of string conversions
  • [FLINK-2893] - Rename recovery configuration keys
  • [FLINK-2895] - Duplicate immutable object creation
  • [FLINK-2897] - Use distinct initial indices for OutputEmitter round-robin
  • [FLINK-2898] - Invert Travis CI build order
  • [FLINK-2902] - Web interface sort tasks newest first
  • [FLINK-2904] - Web interface truncated task counts
  • [FLINK-2932] - Flink quickstart docs should ask users to download from https, not http
  • [FLINK-2936] - ClassCastException when using EventTimeSourceFunction in non-EventTime program
  • [FLINK-2940] - Deploy multiple Scala versions for Maven artifacts
  • [FLINK-2947] - Coloured Scala Shell
  • [FLINK-2949] - Add method 'writeSequencefile' to DataSet
  • [FLINK-2961] - Add support for basic type Date in Table API
  • [FLINK-2962] - Cluster startup script refers to unused variable
  • [FLINK-2966] - Improve the way job duration is reported on web frontend.
  • [FLINK-2974] - Add periodic offset commit to Kafka Consumer if checkpointing is disabled
  • [FLINK-2976] - Save and load checkpoints manually
  • [FLINK-2981] - Update README for building docs
  • [FLINK-2984] - Support lenient parsing of SVMLight input files
  • [FLINK-2991] - Extend Window Operators to Allow Efficient Fold Operation
  • [FLINK-2993] - Set default DelayBetweenExecutionRetries to 0
  • [FLINK-2994] - Client sysout logging does not report exceptions
  • [FLINK-3003] - Add container allocation timeout to YARN CLI
  • [FLINK-3017] - Broken 'Slots' link on Streaming Programming Guide
  • [FLINK-3023] - Show Flink version + commit id for -SNAPSHOT versions in web frontend
  • [FLINK-3028] - Cannot cancel restarting job via web frontend
  • [FLINK-3030] - Enhance Dashboard to show Execution Attempts
  • [FLINK-3039] - Trigger KeyValueState cannot be Scala Int
  • [FLINK-3040] - Add docs describing how to configure State Backends
  • [FLINK-3042] - Define a way to let types create their own TypeInformation
  • [FLINK-3044] - In YARN mode, configure FsStateBackend by default.
  • [FLINK-3045] - Properly expose the key of a kafka message
  • [FLINK-3046] - Integrate the Either Java type with the TypeExtractor
  • [FLINK-3049] - Move "Either" type to package "org.apache.flink.types"
  • [FLINK-3050] - Add custom Exception type to suppress job restarts
  • [FLINK-3051] - Define a maximum number of concurrent inflight checkpoints
  • [FLINK-3055] - ExecutionVertex has duplicate method getParallelSubtaskIndex and getSubTaskIndex
  • [FLINK-3056] - Show bytes sent/received as MBs/GB and so on in web interface
  • [FLINK-3063] - [py] Remove combiner
  • [FLINK-3069] - Make state materialization asynchronous
  • [FLINK-3073] - Activate streaming mode by default
  • [FLINK-3074] - Make ApplicationMaster/JobManager akka port configurable
  • [FLINK-3076] - Display channel exchange mode in the plan visualizer
  • [FLINK-3077] - Add "version" command to CliFrontend for showing the version of the installation
  • [FLINK-3083] - Add docs how to configure streaming fault tolerance
  • [FLINK-3084] - File State Backend should not write very small state into files
  • [FLINK-3085] - Move State Backend Initialization from "registerInputOutput()" to "invoke()"
  • [FLINK-3114] - Read cluster's default parallelism upon remote job submission
  • [FLINK-3115] - Update Elasticsearch connector to 2.X
  • [FLINK-3116] - Remove RecordOperator
  • [FLINK-3120] - Set number of event loop thread and number of buffer pool arenas to same number
  • [FLINK-3122] - Generalize value type in LabelPropagation
  • [FLINK-3123] - Allow setting custom start-offsets for the Kafka consumer
  • [FLINK-3124] - Introduce a TaskInfo object to better represent task name, index, attempt number etc.
  • [FLINK-3127] - Measure backpressure in Flink jobs
  • [FLINK-3131] - Expose checkpoint metrics
  • [FLINK-3132] - Restructure streaming guide
  • [FLINK-3133] - Introduce collect()/coun()/print() methods in DataStream API
  • [FLINK-3135] - Add chainable driver for UNARY_NO_OP strategy
  • [FLINK-3147] - HadoopOutputFormatBase should expose fields as protected
  • [FLINK-3154] - Update Kryo version from 2.24.0 to 3.0.3
  • [FLINK-3160] - Aggregate operator statistics by TaskManager
  • [FLINK-3161] - Externalize cluster start-up and tear-down when available
  • [FLINK-3169] - Drop type from Record Data Model
  • [FLINK-3172] - Specify jobmanager port in HA mode
  • [FLINK-3174] - Add merging WindowAssigner
  • [FLINK-3176] - Window Apply Website Example
  • [FLINK-3178] - Make Closing Behavior of Window Operators Configurable
  • [FLINK-3181] - The vertex-centric SSSP example and library method send unnecessary messages during the first superstep
  • [FLINK-3194] - Remove web client
  • [FLINK-3198] - Rename Grouping.getDataSet() method and add JavaDocs
  • [FLINK-3200] - Use Partitioned State Abstraction in WindowOperator
  • [FLINK-3209] - Remove Unused ProcessingTime, EventTime and AbstractTime
  • [FLINK-3213] - Union of two streams with different parallelism should adjust parallelism automatically
  • [FLINK-3219] - Implement DataSet.count using a single operator
  • [FLINK-3232] - Add option to eagerly deploy channels
  • [FLINK-3233] - PartitionOperator does not support expression keys on atomic types
  • [FLINK-3234] - SortPartition does not support KeySelectorFunctions
  • [FLINK-3235] - Drop Flink-on-Tez code
  • [FLINK-3244] - Add log messages to savepoint coordinator restore
  • [FLINK-3246] - Consolidate maven project names with *-parent suffix
  • [FLINK-3249] - Wrong "unknown partition state/input gate" error messages
  • [FLINK-3258] - Merge AbstractInvokable's registerInputOutput and invoke
  • [FLINK-3259] - Redirect programming guides to new layout
  • [FLINK-3262] - Remove fuzzy versioning from Bower dependencies
  • [FLINK-3265] - RabbitMQ Source not threadsafe: ConcurrentModificationException
  • [FLINK-3270] - Add example for reading and writing to Kafka
  • [FLINK-3273] - Remove Scala dependency from flink-streaming-java
  • [FLINK-3276] - Move runtime parts of flink-streaming-java to flink-runtime
  • [FLINK-3278] - Add Partitioned State Backend Based on RocksDB
  • [FLINK-3290] - [py] Generalize OperationInfo transfer
  • [FLINK-3299] - Remove ApplicationID from Environment
  • [FLINK-3303] - Move all non-batch specific classes in flink-java to flink-core
  • [FLINK-3305] - Remove JavaKaffee serializer util
  • [FLINK-3306] - Auto type registration at Kryo is buggy
  • [FLINK-3308] - [py] Remove debug mode
  • [FLINK-3309] - [py] Resolve maven warnings
  • [FLINK-3310] - Add back pressure statistics to web frontend
  • [FLINK-3334] - Change default log4j.properties Conversion pattern to include year-month-day in the timestamp
  • [FLINK-3336] - Add Semi-Rebalance Data Shipping for DataStream
  • [FLINK-3347] - TaskManager ActorSystems need to restart themselves in case they notice quarantine
  • [FLINK-3348] - taskmanager.memory.off-heap missing bc documentation
  • [FLINK-3355] - Allow passing RocksDB Option to RocksDBStateBackend
  • [FLINK-3358] - Create constructors for FsStateBackend in RocksDBBackens
  • [FLINK-3364] - Don't initialize SavepointStore in JobManager constructor
  • [FLINK-3375] - Allow Watermark Generation in the Kafka Source
  • [FLINK-3376] - Add an illustration of Event Time and Watermarks to the docs
  • [FLINK-3378] - Consolidate TestingCluster and FokableFlinkMiniCluster
  • [FLINK-3379] - Refactor TimestampExtractor
  • [FLINK-3386] - Kafka consumers are not properly respecting the "auto.offset.reset" behavior
  • [FLINK-3389] - Add Pre-defined Options settings for RocksDB State backend
  • [FLINK-3395] - Polishing the web UI
  • [FLINK-3402] - Refactor Common Parts of Stream/Batch Documentation
  • [FLINK-3413] - Remove implicit Seq to DataStream conversion
  • [FLINK-3419] - Drop partitionByHash from DataStream
  • [FLINK-3421] - Remove all unused ClassTag context bounds in the Streaming Scala API
  • [FLINK-3422] - Scramble HashPartitioner hashes
  • [FLINK-3430] - Remove "no POJO" warning in TypeAnalyzer
  • [FLINK-3435] - Change interplay of Ingestion Time and Event Time
  • [FLINK-3436] - Remove ComplexIntegrationITCase
  • [FLINK-3455] - Bump Kafka 0.9 connector dependency to Kafka 0.9.0.1
  • [FLINK-3459] - Make build SBT compatible
  • [FLINK-3460] - Make flink-streaming-connectors' flink dependencies provided
  • [FLINK-3461] - Remove duplicate condition check in ZooKeeperLeaderElectionService
  • [FLINK-3469] - Improve documentation for grouping keys
  • [FLINK-3509] - Update Hadoop versions in release script and on travis to the latest minor version
  • [FLINK-3520] - Periodic watermark operator should emit current watermark in close()
  • [FLINK-3521] - Make Iterable part of method signature for WindowFunction
  • [FLINK-3522] - Storm examples "PrintSampleStream" throws an error if called without arguments
  • [FLINK-3527] - Scala DataStream has no transform method
  • [FLINK-3528] - Add Incremental Fold for Non-Keyed Window Operator
  • [FLINK-3535] - Decrease logging verbosity of StackTraceSampleCoordinator
  • [FLINK-3536] - Make clearer distinction between event time and processing time
  • [FLINK-3538] - DataStream join API does not enforce consistent usage
  • [FLINK-3548] - Remove unnecessary generic parameter from SingleOutputStreamOperator
  • [FLINK-3559] - Don't print pid file check if no active PID

New Feature

  • [FLINK-1723] - Add cross validation for model evaluation
  • [FLINK-2157] - Create evaluation framework for ML library
  • [FLINK-2390] - Replace iteration timeout with algorithm for detecting termination
  • [FLINK-2411] - Add basic graph summarization algorithm
  • [FLINK-2435] - Add support for custom CSV field parsers
  • [FLINK-2720] - Add Storm-CountMetric in flink-stormcompatibility
  • [FLINK-2728] - Add missing features to new web dashboard
  • [FLINK-2871] - Add OuterJoin strategy with HashTable on outer side
  • [FLINK-2905] - Add intersect method to Graph class
  • [FLINK-2951] - Add Union operator to Table API.
  • [FLINK-2955] - Add operations introduction in Table API page.
  • [FLINK-2978] - Integrate web submission interface into the new dashboard
  • [FLINK-2996] - Add config entry to define BlobServer port
  • [FLINK-3001] - Add Support for Java 8 Optional type
  • [FLINK-3002] - Add an EitherType to the Java API
  • [FLINK-3058] - Add Kafka consumer for new 0.9.0.0 Kafka API
  • [FLINK-3093] - Introduce annotations for interface stability
  • [FLINK-3102] - Allow reading from multiple topics with one FlinkKafkaConsumer
  • [FLINK-3170] - Expose task manager metrics via JMX
  • [FLINK-3192] - Add explain support to print ast and sql physical execution plan.

Task

  • [FLINK-1681] - Remove the old Record API
  • [FLINK-2973] - Add flink-benchmark with compliant licenses again
  • [FLINK-3112] - Remove unused RecordModelPostPass class
  • [FLINK-3113] - Remove unused global order methods from GenericDataSinkBase
  • [FLINK-3155] - Update Flink docker version to latest stable Flink version
  • [FLINK-3186] - Deprecate DataSink.sortLocalOutput() methods
  • [FLINK-3366] - Rename @Experimental annotation to @PublicEvolving
  • [FLINK-3367] - Annotate all user-facing API classes with @Public or @PublicEvolving
  • [FLINK-3484] - Add setSlotSharingGroup documentation
  • [FLINK-3490] - Bump Chill version to 0.7.4

Test

  • [FLINK-2480] - Improving tests coverage for org.apache.flink.streaming.api
  • [FLINK-2573] - Add Kerberos test case
  • [FLINK-2803] - Add test case for Flink's memory allocation
  • [FLINK-3517] - Number of job and task managers not checked in scripts

Wish

  • [FLINK-2429] - Remove the "enableCheckpointing()" without interval variant
  • [FLINK-2995] - Set default number of retries to larger than 0
  • [FLINK-3377] - Remove final flag from ResultPartitionWriter class

Known Issues

  • The mvn clean verify command does not succeed for the source release, because of issues with the Maven shade plugin. This has been resolved in the release-1.0 branch.
  • FLINK-3578 - Scala DataStream API does not support Rich Window Functions.

Copyright © 2014-2016 The Apache Software Foundation. All Rights Reserved.

Apache Flink, Apache, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation.

Privacy Policy · RSS feed