How to set PKCS1 v1.5 padding to RSA in Dart? - flutter

Question

I want to encrypt the value with public key, I used pointycastle package and RSA and PKCS1 v1.5 padding, but this didn't work.
publicKey: 7/ZVKLxKXsQnlVUozywoumbFGbSz8kbH178dT165ODd/a/dUMAsdvs90iCLZo4KVoxZbQpD9d5K/Qgs1wjjylQ==
plainText:
a57b4b30-6b96-11ec-a8cf-c308568a983a
String encrypt(String plaintext, String publicKey) {
var modulusBytes = base64.decode(publicKey);
var modulus = BigInt.parse(hex.encode(modulusBytes), radix: 16);
var exponent = BigInt.parse(hex.encode(base64.decode('AQAB')), radix: 16);
var engine = RSAEngine()
..init(
true,
PublicKeyParameter<RSAPublicKey>(RSAPublicKey(modulus, exponent)),
);
//PKCS1.5 padding
var k = modulusBytes.length;
var plainBytes = utf8.encode(plaintext);
var paddingLength = k - 3 - plainBytes.length;
var eb = Uint8List(paddingLength + 3 + plainBytes.length);
var r = Random.secure();
eb.setRange(paddingLength + 3, eb.length, plainBytes);
eb[0] = 0;
eb[1] = 2;
eb[paddingLength + 2] = 0;
for (int i = 2; i < paddingLength + 2; i++) {
eb[i] = r.nextInt(254) + 1;
}
print(plainBytes.length);
print(eb);
return base64.encode(
engine.process(eb),
);
}

Answer 1

A couple of comments first...
Let's assume that the "public key" you provide is the modulus of an RSA key. We have to assume that the exponent is 65537. You'd normally want to get the public key in a PEM or similar. If that is the modulus, it's painfully short at just 512 bytes.
There only version of PKCS1 that's less than 2.x is 1.5. In particular, pointy castle's implementation in PKCS1Encoding is an implementation of 1.5.
You can only encrypt bytes. So, how are we meant to turn your "plaintext" into bytes. It could be a string that we could just encode in utf8. But it looks like a UUID. Maybe you are meant to interpret it as its normal representation as a 128 bit number, in bytes.
Finally, the output of the encryption will be bytes. How are you meant to send those to the server? Does it want them converted to a string first? If so, how? In base64? In hex?
The following code assumes: yes, that's a 512 bit modulus; the plaintext is not a UUID that can be interpreted, but is just a string that should be converted to ASCII bytes; the output is just an array of bytes.
import 'dart:convert';
import 'dart:typed_data';
import 'package:convert/convert.dart';
import 'package:pointycastle/export.dart';
void main() {
final publicKey =
'7/ZVKLxKXsQnlVUozywoumbFGbSz8kbH178dT165ODd/a/dUMAsdvs90iCLZo4KVoxZbQpD9d5K/Qgs1wjjylQ==';
final plaintext = 'a57b4b30-6b96-11ec-a8cf-c308568a983a';
var modulusBytes = base64.decode(publicKey);
var modulus = BigInt.parse(hex.encode(modulusBytes), radix: 16);
var exponent = BigInt.from(65537);
var engine = PKCS1Encoding(RSAEngine())
..init(
true,
PublicKeyParameter<RSAPublicKey>(RSAPublicKey(modulus, exponent)),
);
print(engine.process(utf8.encode(plaintext) as Uint8List));
}