You only describe how you generate the certificates but not how you actually use these. How did you import these into the trust store, how did you check if it works, what exact errors you get ...

@SteffenUllrich there's quite a bit of scripts and configuration files to run a simple test case, which is why I pointed it to a repository to avoid the post from cluttering. I'll copy and paste it here then.

updated post for clarity.

@David: I don't think curl on MacOS will use the system CA store. Use --cacert argument instead to specify your CA.

@SteffenUllrich In the end I need my browsers (ios, safari) on various devices to trust the CAs; I had thought curl and python requests requests.get("https://localhost:443") are prerequisite tests I can quickly conduct to check whether it works.

curl --cacert out/CA.crt https://localhost:443 still reproduces the same error. I'm very confused. Is the issue with my CA/server certificate generation, or with the way I add the CA certificate to my OS?

You must not use the same subject for CA certificate and server certificate. This is not specific to MacOS. Apart from this change (i.e. replace CN=special-name with CN=CA) for the CA it works for me on MacOS when testing the generated server certificate with openssl s_server and using curl --cacert ... as client.

@SteffenUllrich You are right, I would never have thought that would be the reason... I distinguished the CN for the CA and server, and now curl works. In fact, curl https://localhost:443 seems to work for me (without need for --cacert). I'm going to investigate this and see if I can reproduce this more clearly.

Strangely, python requests still fails: import requests; requests.get("https://localhost:443").

I guess in the context of the curl problem, this question is answered. However, the wget, requests, and browsers do not recognize the certificate. This probably means that curl has weaker conditions than the latter in validating a server certificate. I'll probably ask a different question in regards to that.

I see, I don't understand the network layers that well. This is unfortunate, then the testing I've been conducting won't really tell me anything about browser-level trust on MacOS/iOS.

Python requests testing can be overcome, by manually adding the path of the CA to the REQUESTS_CA_BUNDLE env variable. On an ios device, however, the only way to trust a self-signed certificate (from my understanding) is to add and trust the root certificate authority. Which is why I want to get the CA generation down.

See apple.stackexchange.com/questions/371725/… for the requirements in iOS. Note also that you specified a validation period of 365 days for the CA but 825 days for the certificate issued by the CA. This makes no sense, the expiration of the certificate should not exceed the expiration of the issuing CA

Also, if you want to know about browser level trust you should test with the relevant browsers. For example Firefox comes with its own trust store. In general, using self-signed certificates or running your own CA is not a good idea if you want to serve many customers and different client applications. It does not really scale and supporting it will be a hassle

Thanks for pointing out the validation issue, I missed that completely.

I'm aware of the self-signed risks. I plan to run this in my internal network behind a vpn, so at all times I would be the only intended user. If I run a public certificate like letsencrypt, I would ideally need to expose my services to the public with a reverse cloud tunnel, which carries its own number of risks, like exposing potentially insecure web servers to the public. In this case, I don't think a self-signed certificate is that dangerous

I also was hoping to learn something about certificate generation (although I wasn't expecting it to be this messy :p) In any case, I finally was able to make an nginx https server that can be accessed by an ios browser, so I guess the problem is effectively solved. I just need to remember and automate this process. Thanks for helping me out!