How to sign data when using pkcs11 engine for openssl with a private key stored on an HSM #92798
Comments
ghost
added
the
|
Tagging subscribers to this area: @dotnet/area-system-security, @bartonjs, @vcsjones Issue DetailsDescriptionHi all, We are working on a way to allow our dotnet api to sign some data using a private key that is stored on an on-premise NShield Connect HSM from Entrust. We are using pkcs11 to communicate with the HSM with a module (
An exception is raised when we call TimelineThe issue is not critical for now, we were already planning to wait for the official release of dotnet 8 before starting to code the new api that will be using this.
We understand full well that we have an on-premise HSM that is not easily accessible for you to test. Reproduction StepsUsing this snippet of code : byte[] data = ...;
using SafeEvpPKeyHandle privateKeyHandle = SafeEvpPKeyHandle.OpenPrivateKeyFromEngine("pkcs11", "pkcs11:type=private;object=my-private-key");
using RSAOpenSsl rsaPri = new(privateKeyHandle);
rsaPri.SignData(data, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1);Expected behaviorThe data is signed using the private key stored into the HSM. Actual behavior
ConfigurationOS
Dotnet
OpenSSL
Key
Other informationFollowing the above stack trace, it brings us here : We did some trial and error to determine where the root cause was :
Please note that we cut the pkcs11 uri in the following samples for brevity.
Using OpenSSLOutput
Everything went well. Using C codeOutput
Let's explain the output a little :
Everything went well for the signature. However,
We need some guidance if there is something that could be done about the
|
|
@chaoshades I've played around with SoftHSM to generate keys for OpenSC's pkcs11 engine and tpm2tss with tpm keys and I've convinced myself this is specific to your pkcs11 engine implementation and not strictly a bug in .NET. I believe your pkcs11 implementation should be setting following flag on the RSA key when creating EVP_PKEY instance: //RSA* rsa = ... (wherever you create it in your pkcs11 code)
RSA_set_flags(rsa, RSA_FLAG_EXT_PKEY);
if some older OpenSSL then:
rsa->flags |= RSA_FLAG_EXT_PKEY;having said that the check we do seems redundant and it seems to be causing issue with at least one implementation - while it provides slightly better exception message when you accidentally pass public key for something expecting private key I'm not sure extra work and extra issue it causes is worth it (and we should probably let OpenSSL decide what's best error to display) so I'll still consider removing HasNoPrivateKey and fixing this issue in .NET. |
|
@krwq We will ask our vendor if they set this flag in their module. About the |
|
Based on the findings above and the anticipation that this issue can be addressed without a change to .NET 8, I'm going to move this issue to the 9.0.0 milestone so that it no longer shows up as blocking the 8.0.0 release. If the end-to-end does still end up being blocked and we find a change is necessary into .NET 8 for the scenario to fully work, we can explore a fix into .NET 9 first with the potential to backport into .NET 8 servicing. Thanks @krwq and @chaoshades for the close collaboration here! |
|
We got feedback from the vendor and the team managing the HSM and the actual pkcs11 engine implementation doesn't set the We also believes a workaround could be possible, like a native call or something that simulate However, our client doesn't want to maintain a fix over .NET 8 for production use. The expertise is quite niche and they prefer to wait rather than risk it. So, moving this issue as non-blocking the .NET 8 release is fine for us. We will continue to follow the issue because as we said above : We will make us available if there is a need to interact or experiment things.
|
|
@chaoshades We implemented OpenSSL Providers support in .NET 9. Have you moved to OpenSSL 3.x by chance? It's unlikely that we'll add new OpenSSL 1.x ENGINE features. |
jeffhandley
added
the
needs-author-action
|
@chaoshades I have fixed this issue in #104961 (will be available in preview 7 or already in nightlies) but only when using OpenSSL 3.x (due to OpenSSL segfaulting on some lower versions we need to continue doing that check and RSA_FLAG_EXT_PKEY is required). For providers this issue will not show up (but that is also due to the fact that they need OpenSSL 3.x). |
Our client didn't upgrade to OpenSSL 3.x because the usage is pretty limited for now.
Oh nice! I will contact the team leader to let them know if they can give me some time to test this. |
dotnet-policy-service
bot
added
needs-further-triage
|
@chaoshades I'm going to go ahead and close this issue since we're not making any further investments into OpenSSL ENGINE support. Hopefully you will be able to proceed with OpenSSL 3.x and Providers. Thanks for reaching out on this topic. |
Description
Hi all,
We are working on a way to allow our dotnet api to sign some data using a private key that is stored on an on-premise NShield Connect HSM from Entrust. We are using pkcs11 to communicate with the HSM with a module (
libcknfast.so) provided by the vendor which is compatible.An exception is raised when we call
SignDataon theRSAOpenSslinstance created from theSafeEvpPKeyHandlereturned by the new dotnet 8 featureOpenPrivateKeyFromEngine.Timeline
The issue is not critical for now, we were already planning to wait for the official release of dotnet 8 before starting to code the new api that will be using this.
We understand full well that we have an on-premise HSM that is not easily accessible for you to test.
We will make us available if there is a need to interact or experiment things.
Reproduction Steps
Using this snippet of code :
Expected behavior
The data is signed using the private key stored into the HSM.
Actual behavior
rsaPri.SignData(...)raises the following exception :Configuration
OS
sh-4.4# cat /etc/os-releaseDotnet
sh-4.4# dotnet --infoOpenSSL
sh-4.4# openssl versionsh-4.4# openssl engine pkcs11 -t -vsh-4.4# cat /etc/pki/tls/openssl.cnfKey
Private Key Pkcs11 Extensions
p11tool --provider /opt/nfast/toolkits/pkcs11/libcknfast.so --list-privkeysOther information
Following the above stack trace, it brings us here :
https://github.com/dotnet/runtime/blob/df8cbd11b84a5930ec71666ad87479d559872718/src/native/libs/System.Security.Cryptography.Native/pal_evp_pkey_rsa.c#L254C1-L263C6
We did some trial and error to determine where the root cause was :
Please note that we cut the pkcs11 uri in the following samples for brevity.
Using OpenSSL
Output
echo -n "Hello, world!" > /tmp/test.txtopenssl dgst -engine pkcs11 -sign "pkcs11:type=private;object=my-private-key" -sha256 -keyform engine -sigopt rsa_padding_mode:pkcs1 -out /tmp/test.sig -binary /tmp/test.txtopenssl dgst -engine pkcs11 -verify "pkcs11:type=public;object=my-public-key" -sha256 -keyform engine -signature /tmp/test.sig -binary /tmp/test.txtEverything went well.
So we believe this would eliminate potential issues that could be on our end with OpenSSL configurations and that the proprietary module is not fiddling with the process.
Using C code
Output
ossl_test.c.txt
build.sh.txt
sh-4.4# ./build.shsh-4.4# ./ossl_testLet's explain the output a little :
Main - Got Private Key!means that the pkcs11 engine was initialized and the private key was loaded accordingly.Got RSA!untilMain - HasNoPrivateKey...are printf used to debug theHasNoPrivateKeyfunction.Main - Got Signature!means the signature was done correctly using the pkcs11 engine.Everything went well for the signature.
However,
HasNoPrivateKeyreturns 1.Which means it looks like there are no private key and then dotnet will raise the exception above :
error:04084093:rsa routines::value missingWe need some guidance if there is something that could be done about the
HasNoPrivateKeyfunction or if there something to do with theRSA_FLAG_EXT_PKEYflag.The text was updated successfully, but these errors were encountered: