Skip to content
Advertisement

Unexpected backlog size in Pulsar

I’m using Pulsar for communication between services and I’m experiencing flakiness in a quite simple test of producers and consumers.

In JUnit 4 test, I spin up (my own wrappers around) a ZooKeeper server, a BookKeeper bookie, and a PulsarService; the configurations should be quite standard.

The test can be summarized in the following steps:

  1. build a producer;
  2. build a consumer (say, a reader of a Pulsar topic);
  3. check the message backlog (using precise backlog);
    • this is done by getting the current subscription via PulsarAdmin#topics#getStats#subscriptions
    • I expect it to be 0, as nothing was sent on the topic, but sometimes it is 1, but this seems another problem…
  4. build a new producer and synchronously send a message onto the topic;
  5. build a new consumer and read the messages on the topic;
    • I expect a backlog of one message, and I actually read one
  6. build a new producer and synchronously send four messages;
  7. fetch again the messages, using the messageID read at step 5 as start message ID;
    • I expect a backlog of four messages here, and most of the time this value is correct, but running the test about ten times I consistently get 2 or 5

I tried debugging the test, but I cannot figure out where those values come from; did I misunderstand something?

Advertisement

Answer

Things you can try if not already done:

Advertisement