java - Kinesis: What is the best/safe way to shutdown a worker? -


i using aws kinesis client library.

i need way shutdown kinesis worker thread during deployments so, stop @ checkpoint , not in middle of processrecords().

i see shutdown boolean present in worker.java made private.

the reason need checkpointing , idempotency critical me , don't want kill process in middle of batch.

[edit]

thanks @captainmurphy, noticed worker.java exposes shutdown() method safely shuts down worker , leasecoordinator. doesn't call shutdown() task in irecordprocessor. abruptly terminates irecordprocessor without worrying state.

i understand idempotency between checkpoints not guaranteed kcl , developer should make design fault tolerant feel irecordprocessor should shutdown before leasecoordinator stops irrespective of that.

the record processor shutdown method indeed called when call shutdown on worker. can trace shutdowntask class, created shardconsumer class, closed worker.

so can checkpoint @ last received record listening shutdown call, passing checkpointer last sequence received function @ iteration point of last processed value. e.g. in overridden processrecords():

for(record currrecord : records) {     someprocesssinglerecordmethod(currrecord)     if(shutdown)      {          checkpointer.checkpoint(currrecord.getsequencenumber());          return;      }  } 

where shutdown method sets shutdown flag true.

note it's still best practice kinesis applications designed in "at least once" fashion in case of ungraceful shutdown, such instance termination. receive , process "just once" may not use case kinesis.


Comments

Popular posts from this blog

shopping cart - Page redirect not working PHP -

php - How to modify a menu to show sub-menus -

python - Installing PyDev in eclipse is failed -