The Apache Flink community released the first bugfix version of the Apache Flink 1.12 series.

This release includes 79 fixes and minor improvements for Flink 1.12.0. The list below includes a detailed list of all fixes and improvements.

We highly recommend all users to upgrade to Flink 1.12.1.

Attention: Using unaligned checkpoints in Flink 1.12.0 combined with two/multiple inputs tasks or with union inputs for single input tasks can result in corrupted state.

This can happen if a new checkpoint is triggered before recovery is fully completed. For state to be corrupted a task with two or more input gates must receive a checkpoint barrier exactly at the same time this tasks finishes recovering spilled in-flight data. In such case this new checkpoint can succeed, with corrupted/missing in-flight data, which will result in various deserialisation/corrupted data stream errors when someone attempts to recover from such corrupted checkpoint.

Using unaligned checkpoints in Flink 1.12.1, a corruption may occur in the checkpoint following a declined checkpoint.

A late barrier of a canceled checkpoint may lead to buffers being not written into the successive checkpoint, such that recovery is not possible. This happens, when the next checkpoint barrier arrives at a given operator before all previous barriers arrived, which can only happen after cancellation in unaligned checkpoints.

Updated Maven dependencies:

<dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-java</artifactId> <version>1.12.1</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-streaming-java_2.11</artifactId> <version>1.12.1</version> </dependency> <dependency> <groupId>org.apache.flink</groupId> <artifactId>flink-clients_2.11</artifactId> <version>1.12.1</version> </dependency>

You can find the binaries on the updated Downloads page.

List of resolved issues:

Sub-task

  • [FLINK-18897] - Add documentation for the maxwell-json format
  • [FLINK-20352] - Rework command line interface documentation page
  • [FLINK-20353] - Rework logging documentation page
  • [FLINK-20354] - Rework standalone deployment documentation page
  • [FLINK-20355] - Rework K8s deployment documentation page
  • [FLINK-20356] - Rework Mesos deployment documentation page
  • [FLINK-20422] - Remove from .html files in flink documentation
  • [FLINK-20485] - Map views are deserialized multiple times
  • [FLINK-20601] - Rework PyFlink CLI documentation

Bug

  • [FLINK-19369] - BlobClientTest.testGetFailsDuringStreamingForJobPermanentBlob hangs
  • [FLINK-19435] - Deadlock when loading different driver classes concurrently using Class.forName
  • [FLINK-19725] - Logger cannot be initialized due to timeout: LoggerInitializationException is thrown
  • [FLINK-19880] - Fix ignore-parse-errors not work for the legacy JSON format
  • [FLINK-20213] - Partition commit is delayed when records keep coming
  • [FLINK-20221] - DelimitedInputFormat does not restore compressed filesplits correctly leading to dataloss
  • [FLINK-20273] - Fix Table api Kafka connector Sink Partitioner Document Error
  • [FLINK-20372] - Update Kafka SQL connector page to mention properties.* options
  • [FLINK-20389] - UnalignedCheckpointITCase failure caused by NullPointerException
  • [FLINK-20404] - ZooKeeper quorum fails to start due to missing log4j library
  • [FLINK-20419] - Insert fails due to failure to generate execution plan
  • [FLINK-20428] - ZooKeeperLeaderElectionConnectionHandlingTest.testConnectionSuspendedHandlingDuringInitialization failed with "No result is expected since there was no leader elected before stopping the server, yet"
  • [FLINK-20429] - KafkaTableITCase.testKafkaTemporalJoinChangelog failed with unexpected results
  • [FLINK-20433] - UnalignedCheckpointTestBase.execute failed with "TestTimedOutException: test timed out after 300 seconds"
  • [FLINK-20464] - Some Table examples are not built correctly
  • [FLINK-20467] - Fix the Example in Python DataStream Doc
  • [FLINK-20470] - MissingNode can't be casted to ObjectNode when deserializing JSON
  • [FLINK-20476] - New File Sink end-to-end test Failed
  • [FLINK-20486] - Hive temporal join should allow monitor interval smaller than 1 hour
  • [FLINK-20492] - The SourceOperatorStreamTask should implement cancelTask() and finishTask()
  • [FLINK-20493] - SQLClientSchemaRegistryITCase failed with "Could not build the flink-dist image"
  • [FLINK-20521] - Null result values are being swallowed by RPC system
  • [FLINK-20525] - StreamArrowPythonGroupWindowAggregateFunctionOperator doesn't handle rowtime and proctime properly
  • [FLINK-20543] - Fix typo in upsert kafka docs
  • [FLINK-20554] - The Checkpointed Data Size of the Latest Completed Checkpoint is incorrectly displayed on the Overview page of the UI
  • [FLINK-20582] - Fix typos in CREATE Statements docs
  • [FLINK-20607] - a wrong example in udfs page.
  • [FLINK-20615] - Local recovery and sticky scheduling end-to-end test timeout with "IOException: Stream Closed"
  • [FLINK-20626] - Canceling a job when it is failing will result in job hanging in CANCELING state
  • [FLINK-20630] - [Kinesis][DynamoDB] DynamoDB Streams Consumer fails to consume from Latest
  • [FLINK-20646] - ReduceTransformation does not work with RocksDBStateBackend
  • [FLINK-20648] - Unable to restore job from savepoint when using Kubernetes based HA services
  • [FLINK-20664] - Support setting service account for TaskManager pod
  • [FLINK-20665] - FileNotFoundException when restore from latest Checkpoint
  • [FLINK-20666] - Fix the deserialized Row losing the field_name information in PyFlink
  • [FLINK-20669] - Add the jzlib LICENSE file in flink-python module
  • [FLINK-20703] - HiveSinkCompactionITCase test timeout
  • [FLINK-20704] - Some rel data type does not implement the digest correctly
  • [FLINK-20756] - PythonCalcSplitConditionRule is not working as expected
  • [FLINK-20764] - BatchGroupedReduceOperator does not emit results for singleton inputs
  • [FLINK-20781] - UnalignedCheckpointITCase failure caused by NullPointerException
  • [FLINK-20784] - .staging_xxx does not exist, when insert into hive
  • [FLINK-20793] - Fix NamesTest due to code style refactor
  • [FLINK-20803] - Version mismatch between spotless-maven-plugin and google-java-format plugin
  • [FLINK-20841] - Fix compile error due to duplicated generated files

Improvement

  • [FLINK-19013] - Log start/end of state restoration
  • [FLINK-19259] - Use classloader release hooks with Kinesis producer to avoid metaspace leak
  • [FLINK-19832] - Improve handling of immediately failed physical slot in SlotSharingExecutionSlotAllocator
  • [FLINK-20055] - Datadog API Key exposed in Flink JobManager logs
  • [FLINK-20168] - Translate page 'Flink Architecture' into Chinese
  • [FLINK-20209] - Add missing checkpoint configuration to Flink UI
  • [FLINK-20298] - Replace usage of in flink documentation
  • [FLINK-20468] - Enable leadership control in MiniCluster to test JM failover
  • [FLINK-20510] - Enable log4j2 monitor interval by default
  • [FLINK-20519] - Extend HBase notice with transitively bundled dependencies
  • [FLINK-20570] - The NOTE tip style is different from the others in process_function page.
  • [FLINK-20588] - Add docker-compose as appendix to Mesos documentation
  • [FLINK-20629] - [Kinesis][EFO] Migrate from DescribeStream to DescribeStreamSummary
  • [FLINK-20647] - Use yield to generate output datas in ProcessFunction for Python DataStream
  • [FLINK-20650] - Mark "native-k8s" as deprecated in docker-entrypoint.sh
  • [FLINK-20651] - Use Spotless/google-java-format for code formatting/enforcement
  • [FLINK-20682] - Add configuration options related to hadoop
  • [FLINK-20697] - Correct the Type of "lookup.cache.ttl" in jdbc.md/jdbc.zh.md
  • [FLINK-20790] - Generated classes should not be put under src/ directory
  • [FLINK-20792] - Allow shorthand invocation of spotless
  • [FLINK-20805] - Blink runtime classes partially ignored by spotless
  • [FLINK-20822] - Don't check whether a function is generic in hive catalog
  • [FLINK-20866] - Add how to list jobs in Yarn deployment documentation when HA enabled

Task

  • [FLINK-20300] - Create Flink 1.12 release notes
  • [FLINK-20906] - Update copyright year to 2021 for NOTICE files

See All 111 Episodes of "Flink Blog Feed"