Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-1653

Error when using PubsubIO with the DirectRunner

Details

    • Bug
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • 0.5.0
    • 2.0.0
    • io-java-gcp, runner-direct
    • None

    Description

      Reading a source with a `org.apache.beam.sdk.io.PubsubIO` and running the pipeline with `DirectRunner` result in an error.

      The error occur usually after reading ~200 messages from pubsub at the moment where `PubsubIO` try to ack the messages.

      The error:
      ```
      Exception in thread "main" java.lang.IllegalStateException: Cannot finalize a restored checkpoint
      at org.apache.beam.sdk.repackaged.com.google.common.base.Preconditions.checkState(Preconditions.java:174)
      at org.apache.beam.sdk.io.PubsubUnboundedSource$PubsubCheckpoint.finalizeCheckpoint(PubsubUnboundedSource.java:293)
      at org.apache.beam.runners.direct.UnboundedReadEvaluatorFactory$UnboundedReadEvaluator.finishRead(UnboundedReadEvaluatorFactory.java:205)
      at org.apache.beam.runners.direct.UnboundedReadEvaluatorFactory$UnboundedReadEvaluator.processElement(UnboundedReadEvaluatorFactory.java:142)
      at org.apache.beam.runners.direct.TransformExecutor.processElements(TransformExecutor.java:139)
      at org.apache.beam.runners.direct.TransformExecutor.run(TransformExecutor.java:107)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)
      ```

      Attachments

        Issue Links

          Activity

            People

              tgroh Thomas Groh
              dysosmus FĂ©lix Mattrat
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: