Documentation feedback about redpanda HOT 14 CLOSED

hey @bithavoc we are working on acls right now and part of that is to tackle authentication. today we only have ssl + plain, but by the end of the december we expect to have sasl+scram in addition to plain, so you could have a common SSL+SCRAM (user and password) setup.

we'll keep working on the docs, and please don't hesitate to dump more feedback here, it's great!

from redpanda.

Sorry I keep hijacking this thread with my experience, I know this issue was originally about about documentation but maybe this is somewhat related.

First, I have Redpanda running 🚀 yay!, I was able to use rpk to produce and consume dummy messages, but even when my app is ready to use any kafka cluster I still can't use Redpanda.

Here's my situation: given I have no knowledge of how to operate a redpanda cluster and there is no docker image, I'm being forced to use a dedicated server to act as a single-node redpanda cluster, I'm running it in a VM with VM-level backups so I can restore the entire thing without reconstructing my cluster. Also, my kafka app is in a kubernetes cluster in a different cloud, no VPC peering available nor I would to do it if available, so I need my app to connect to my redpanda instance via public traffic but the default setup doesn't include and tls configuration for kafka_api, hence my suggestion: the DEV package should at least automatically generate a self-signed TLS and configure it for me so I can connect securely from anywhere just by opening the right ports in the firewall.

A rpk cert command in a similar spirit to cockroach's cert command to create client certs would be even better but not ideal, I think cockroach's approach is a bit too much due the hassle of creating and maintaining client certificates when deploying apps and clusters from scratch for testing and reproducible environments, etc

IMO both developer and production mode, at minimum, should work as follows:

• kafka_api_tls.require_client_auth: false
• SASL/PLAIN? ( I just want to use User and Password with a self-signed CA, no client verification required)

This is where the documentation could shine:

• Authentication Mechanisms supported by Redpanda
• How to properly setup TLS, it's automatic by default, but just in case the user wants to do it or rotate secrets, then describe how to generate files for kafka_api_tls.* with openssl, cfssl, etc

btw, in this page it seems like kafka_api_tls is duplicated.

from redpanda.

cc: @rkruze @yougotashovel

from redpanda.

Thanks for the feedback @bithavoc! We'll get on this asap.

from redpanda.

@0x5d i think @dotnwat uses area/<thing> and has a color coding scheme. might be worth checking w/ him.

from redpanda.

Oh, that label was already there so I woke it up from its slumber. I'll create a new one.

from redpanda.

Also, I think the same place where the docs invite the visitors to run one of those package one-liners should also mention RAM requirements, when I tried the DEB script in Ubuntu 20 LTS in a node with 2GB of RAM I get:

Job for redpanda.service failed because the control process exited with error code.
See "systemctl status redpanda.service" and "journalctl -xe" for details.

logs say:

Dec 01 00:14:32 test-red-panda rpk[2683]: INFO  2020-12-01 00:14:32,774 [shard 0] syschecks - Writing pid file "/var/lib/redpanda/data/pid.lock"
Dec 01 00:14:32 test-red-panda rpk[2683]: ERROR 2020-12-01 00:14:32,774 [shard 0] syschecks - Memory: '499122176' below recommended: '1073741824'
Dec 01 00:14:32 test-red-panda rpk[2683]: INFO  2020-12-01 00:14:32,775 [shard 0] redpanda::main - application.cc:88 - Failure during startup: std::runtime_error (Memory: '499122176' below recommended: '1073741824')
Dec 01 00:14:32 test-red-panda systemd[1]: redpanda.service: Main process exited, code=exited, status=1/FAILURE
Dec 01 00:14:32 test-red-panda systemd[1]: redpanda.service: Failed with result 'exit-code'.
Dec 01 00:14:32 test-red-panda systemd[1]: Failed to start Redpanda, the fastest queue in the West..

So there's a minimum of memory but no mention anywhere in the docs, I'm sorry if I missed it if it's there, the only requirements I could see in the instructions page was something about XFS and port 9092, nothing else.

You may be wondering why I'm trying redpanda in a VM with 2GB of RAM, well, it's my perception that given this project was built in C++ and not Java, then the minimal requirements would be more forgiving, I remember running Kafka in Docker in a VM with similar specs, it was painful due the non root user requirements in the bitnami kafka images but is totally doable.

from redpanda.

i think that's a good point. @0x5d - should we default to 1core ?

from redpanda.

I'm all about making the first impression as seamless as possible. I think @rkruze had also mentioned that the dev mode should be the default. If we make that the case, then redpanda.developer_mode would be true and the minimum memory check wouldn't be strict. I think that's the way to go.

from redpanda.

Fully agree that redpanda.developer_mode should be true.

from redpanda.

@dotnwat It seems like some of the issues here have been addressed. Can you review it and pick out remaining action items?

btw, in this page it seems like kafka_api_tls is duplicated.

This was resolved.

from redpanda.

@bmansheim i think we can close this now. I split out the idea for the compatibility table into a separate ticket to keep things simpler. everything else seemed ok.

from redpanda.

Also, I think the same place where the docs invite the visitors to run one of those package one-liners should also mention RAM requirements, when I tried the DEB script in Ubuntu 20 LTS in a node with 2GB of RAM I get:

Job for redpanda.service failed because the control process exited with error code.
See "systemctl status redpanda.service" and "journalctl -xe" for details.

logs say:

Dec 01 00:14:32 test-red-panda rpk[2683]: INFO  2020-12-01 00:14:32,774 [shard 0] syschecks - Writing pid file "/var/lib/redpanda/data/pid.lock"
Dec 01 00:14:32 test-red-panda rpk[2683]: ERROR 2020-12-01 00:14:32,774 [shard 0] syschecks - Memory: '499122176' below recommended: '1073741824'
Dec 01 00:14:32 test-red-panda rpk[2683]: INFO  2020-12-01 00:14:32,775 [shard 0] redpanda::main - application.cc:88 - Failure during startup: std::runtime_error (Memory: '499122176' below recommended: '1073741824')
Dec 01 00:14:32 test-red-panda systemd[1]: redpanda.service: Main process exited, code=exited, status=1/FAILURE
Dec 01 00:14:32 test-red-panda systemd[1]: redpanda.service: Failed with result 'exit-code'.
Dec 01 00:14:32 test-red-panda systemd[1]: Failed to start Redpanda, the fastest queue in the West..

So there's a minimum of memory but no mention anywhere in the docs, I'm sorry if I missed it if it's there, the only requirements I could see in the instructions page was something about XFS and port 9092, nothing else.

You may be wondering why I'm trying redpanda in a VM with 2GB of RAM, well, it's my perception that given this project was built in C++ and not Java, then the minimal requirements would be more forgiving, I remember running Kafka in Docker in a VM with similar specs, it was painful due the non root user requirements in the bitnami kafka images but is totally doable.

How do you resolve the above problem...? Because i am beginner to this and facing the same issues in redpanda ..

from redpanda.

@sanjaynv - it's mentioned in the thread. see developer_mode:true

from redpanda.