Hello,
I hope not to scare you with this long post. I'm testing Solr (read solar) and it works fine. I have an issue with the DataImport handler, it's set to get data from a MariaDB database, which works fine if I save the database credentials in plain text in the data-config.xml file for the Solr core:
<dataSource type="JdbcDataSource"
driver="org.mariadb.jdbc.Driver"
url="jdbc:mariadb://localhost:3306/DB_TO_QUERY_HERE"
user="USER"
password="PASSWORD"/>Following the documentation at:
I see I can encrypt the password, in order to do so I have to change the above configuration to this:
<dataSource type="JdbcDataSource"
driver="org.mariadb.jdbc.Driver"
url="jdbc:mariadb://localhost:3306/DB_TO_QUERY_HERE"
user="USER"
password="ENCRYPTED_PASSWORD"
encryptKeyFile="/absolute/path/to/key.txt"/>And run a command like this to get the encrypted password:
echo "hello" > pwd.txt
echo "onetestkey" > key.txt
openssl enc -p -aes-128-cbc -a -salt -in pwd.txt -pass file:./key.txt -out pwd.encThe documentation says:
When the password is encrypted, you must provide an extra attribute
encryptKeyFile="/location/of/encryptionkey". This file should a text file with a single line containing the encrypt/decrypt password.
so, for me key.txt is the same file loaded by Solr and by running the above command the pwd.enc file gets populated by:
U2FsdGVkX18xlQ6Mgtirfe7thJ3J17lUC9nWvDf9iU0=Which is a base64 string, decoded it reveals a prefix Salted__ followed by the encrypted password binary blob. And it is the value for the password property in the dataSource tag. The command with enc -p returns more info:
salt=31950E8C82D8AB7D
key=6F489EE571236EC11CCD4ABB5A2DA3A3
iv =99907BAF93567BD934A03E9D720EE020If I run the opposite command, to decrypt and verify the result, it works fine:
openssl enc -base64 -d -aes-128-cbc -in pwd.enc -pass file:./key.txtreturns hello as expected. However, when I attempt to run the DataImport handler in Solr I get this exception:
Full Import failed:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: Error decoding password Processing Document # 1
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:271)
at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:415)
at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:474)
at org.apache.solr.handler.dataimport.DataImporter.lambda$runAsync$0(DataImporter.java:457)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: Error decoding password Processing Document # 1
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:417)
at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:330)
at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:233)
... 4 more
Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: Error decoding password Processing Document # 1
at org.apache.solr.handler.dataimport.JdbcDataSource.decryptPwd(JdbcDataSource.java:132)
at org.apache.solr.handler.dataimport.JdbcDataSource.init(JdbcDataSource.java:75)
at org.apache.solr.handler.dataimport.DataImporter.getDataSourceInstance(DataImporter.java:388)
at org.apache.solr.handler.dataimport.ContextImpl.getDataSource(ContextImpl.java:100)
at org.apache.solr.handler.dataimport.SqlEntityProcessor.init(SqlEntityProcessor.java:53)
at org.apache.solr.handler.dataimport.EntityProcessorWrapper.init(EntityProcessorWrapper.java:77)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:434)
at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:415)
... 6 more
Caused by: java.lang.IllegalStateException: Bad password, algorithm, mode or padding; no salt, wrong number of iterations or corrupted ciphertext.
at org.apache.solr.util.CryptoKeys.decodeAES(CryptoKeys.java:249)
at org.apache.solr.util.CryptoKeys.decodeAES(CryptoKeys.java:195)
at org.apache.solr.handler.dataimport.JdbcDataSource.decryptPwd(JdbcDataSource.java:130)
... 13 more
Caused by: javax.crypto.BadPaddingException: Given final block not properly padded
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:989)
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:845)
at com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:446)
at javax.crypto.Cipher.doFinal(Cipher.java:2165)
at org.apache.solr.util.CryptoKeys.decodeAES(CryptoKeys.java:245)
... 15 moreIn practice it says there is an error decoding the password. The decryptPwd() method is this:
private Properties decryptPwd(Context context, Properties initProps) {
String encryptionKey = initProps.getProperty("encryptKeyFile");
if (initProps.getProperty("password") != null && encryptionKey != null) {
// this means the password is encrypted and use the file to decode it
try {
try (Reader fr = new InputStreamReader(new FileInputStream(encryptionKey), UTF_8)) {
char[] chars = new char[100];//max 100 char password
int len = fr.read(chars);
if (len < 6)
throw new DataImportHandlerException(SEVERE, "There should be a password of length 6 atleast " + encryptionKey);
Properties props = new Properties();
props.putAll(initProps);
String password = null;
try {
password = CryptoKeys.decodeAES(initProps.getProperty("password"), new String(chars, 0, len)).trim();
} catch (SolrException se) {
throw new DataImportHandlerException(SEVERE, "Error decoding password", se.getCause());
}
props.put("password", password);
initProps = props;
}
} catch (IOException e) {
throw new DataImportHandlerException(SEVERE, "Could not load encryptKeyFile " + encryptionKey);
}
}
return initProps;
}and is defined in:
solr-7.0.1/solr/contrib/dataimporthandler/src/java/org/apache/solr/handler/dataimport/JdbcDataSource.javathe BadPaddingException thrown by the decodeAES() method is defined inside:
solr-7.0.1/solr/core/src/java/org/apache/solr/util/CryptoKeys.javaAnd looks like this:
public static String decodeAES(String base64CipherTxt, String pwd) {
int[] strengths = new int[]{256, 192, 128};
Exception e = null;
for (int strength : strengths) {
try {
return decodeAES(base64CipherTxt, pwd, strength);
} catch (Exception exp) {
e = exp;
}
}
throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Error decoding ", e);
}
public static String decodeAES(String base64CipherTxt, String pwd, final int keySizeBits) {
final Charset ASCII = Charset.forName("ASCII");
final int INDEX_KEY = 0;
final int INDEX_IV = 1;
final int ITERATIONS = 1;
final int SALT_OFFSET = 8;
final int SALT_SIZE = 8;
final int CIPHERTEXT_OFFSET = SALT_OFFSET + SALT_SIZE;
try {
byte[] headerSaltAndCipherText = Base64.base64ToByteArray(base64CipherTxt);
// --- extract salt & encrypted ---
// header is "Salted__", ASCII encoded, if salt is being used (the default)
byte[] salt = Arrays.copyOfRange(
headerSaltAndCipherText, SALT_OFFSET, SALT_OFFSET + SALT_SIZE);
byte[] encrypted = Arrays.copyOfRange(
headerSaltAndCipherText, CIPHERTEXT_OFFSET, headerSaltAndCipherText.length);
// --- specify cipher and digest for evpBytesTokey method ---
Cipher aesCBC = Cipher.getInstance("AES/CBC/PKCS5Padding");
MessageDigest md5 = MessageDigest.getInstance("MD5");
// --- create key and IV ---
// the IV is useless, OpenSSL might as well have use zero's
final byte[][] keyAndIV = evpBytesTokey(
keySizeBits / Byte.SIZE,
aesCBC.getBlockSize(),
md5,
salt,
pwd.getBytes(ASCII),
ITERATIONS);
SecretKeySpec key = new SecretKeySpec(keyAndIV[INDEX_KEY], "AES");
IvParameterSpec iv = new IvParameterSpec(keyAndIV[INDEX_IV]);
// --- initialize cipher instance and decrypt ---
aesCBC.init(Cipher.DECRYPT_MODE, key, iv);
byte[] decrypted = aesCBC.doFinal(encrypted);
return new String(decrypted, ASCII);
} catch (BadPaddingException e) {
// AKA "something went wrong"
throw new IllegalStateException(
"Bad password, algorithm, mode or padding;" +
" no salt, wrong number of iterations or corrupted ciphertext.", e);
} catch (IllegalBlockSizeException e) {
throw new IllegalStateException(
"Bad algorithm, mode or corrupted (resized) ciphertext.", e);
} catch (GeneralSecurityException e) {
throw new IllegalStateException(e);
}
}I tested also by setting 6F489EE571236EC11CCD4ABB5A2DA3A3 which was the output of openssl for the key, but I get the same error. I restarted Solr at each change and also attempted to set a wrong path to see if it output a different error and it sorted Could not load encryptKeyFile... as expected. Is there something evidently wrong in what I'm doing? Do you have any suggestions?
Something that bothers me is the exception Caused by: javax.crypto.BadPaddingException: Given final block not properly padded message, which seems to lead to:
The thing about padding errors is that they are usually not errors in the padding but the wrong key or IV during decryption.
And the comment in the decodeAES() method:
// the IV is useless, OpenSSL might as well have use zero's
Above, instead, I see that openssl is defining IV with value 99907BAF93567BD934A03E9D720EE020 so I wonder if the problem is this. And if this can be caused by my system implementation of openssl (Ubuntu 16.04 right now).
Thank you for reading.
