I would argue that Kinesis is not the way to go for quick POC unless you're tied to the JVM.
Pulsar is still niche in most enterprise.
Kafka is not dead, there are many enterprises (including 2 successful ones I've worked at in the past 5 years) that have built POCs and successful products on Kafka. Its supports all language performantly and has tons of community support. I would argue there is nothing better to build a POC on.
Because the official client from AWS is written in Java.
It is possible to write clients in any language, however, it is not that simple especially when you need to handle logic of scalling out or in your kinesis stream (that will split or join shards) and when you have multiple consumers in the same consumer group (you will need a distributed locking mechanism and a logic to steal locks if one consumer dies).
The client handles horizontal scalling, checkpoiting, shards split and shards merges. Using just the SDK, you have to build this yourself (unless you are using Kinesis for use cases that dont need it to be done correctly).
And in the second paragraph of this documentation:
"These examples discuss the Kinesis Data Streams API and use the AWS SDK for Java to get data from a stream. However, for most use cases, you should prefer using the Kinesis Client Library (KCL) . For more information, see Developing KCL 1.x Consumers."
Kafka is just starting to be free of some of its fundamental shortcomings that hindered it's adoption, similar to mongo in the early days, I think. Anyway, I see it being used more widely these days for streaming data.
Apache Pulsar definitely has a better architecture, separating the ingestion from the storage, but once something starts to gain widespread adoption (like Kafka) that lives little room for alternatives.
Some time ago I was talking to some bank software engineers and they loved mongodb because the schemaless nature let them avoid all the red tape which DBAs had created within the organization. Mongodb solves an organizational failure more than anything else.
I deal with text+discrete data. On the one hand this falls well into a document structure, on the other hand it wasn’t clear what exactly was offered on top of what we get from our current setup of elastic+rdbms (we are ok with data duplication and records are ~immutable).
Honestly I don’t have a clear recollection. Their point was to impress on us broad adoption so big names were dropped. This was in the context of Mongo Atlas, their cloud based solution, and how it is being used in heavily regulated/strong privacy environments.
Not nearly as broken as it was. It offers reasonable performance as a document database for unstructured data and surprisingly, works pretty well when containerized.
Kafka's fine, it's got a lot of market share and existing orgs with large deployments aren't going to go through the pain of switching needlessly.
Pulsar is great, I love how it combines message queue and pub-sub semantics. Tiered storage etc. The built-in cluster replication is brilliant, although it has some limitations - you can't replicate from cluster A to cluster B to cluster C due to how Pulsar avoids the active/active infinite replication circle of doom.
But, it's early days for it. Anyone adopting it will need to invest more time in learning the innards compared to Kafka.
If you're not already technically chained into it and Confluence hasn't already upsold your poor organization avoid it.
If you want the early flexibility and the rapid PoC just look at AWS Kinesis/Firehouse.
If you're looking at large scale (+1 gbit ingest, 100k/s, kind of stuff) then Apache Pulsar is where to go.