Hello,

I hope not to scare you with this long post. I'm testing Solr (read solar) and it works fine. I have an issue with the DataImport handler, it's set to get data from a MariaDB database, which works fine if I save the database credentials in plain text in the data-config.xml file for the Solr core:

<dataSource type="JdbcDataSource"
          driver="org.mariadb.jdbc.Driver"
          url="jdbc:mariadb://localhost:3306/DB_TO_QUERY_HERE"
          user="USER"
          password="PASSWORD"/>

Following the documentation at:

I see I can encrypt the password, in order to do so I have to change the above configuration to this:

<dataSource type="JdbcDataSource"
      driver="org.mariadb.jdbc.Driver"
      url="jdbc:mariadb://localhost:3306/DB_TO_QUERY_HERE"
      user="USER"
      password="ENCRYPTED_PASSWORD"
      encryptKeyFile="/absolute/path/to/key.txt"/>

And run a command like this to get the encrypted password:

echo "hello" > pwd.txt
echo "onetestkey" > key.txt
openssl enc -p -aes-128-cbc -a -salt -in pwd.txt -pass file:./key.txt -out pwd.enc

The documentation says:

When the password is encrypted, you must provide an extra attribute encryptKeyFile="/location/of/encryptionkey". This file should a text file with a single line containing the encrypt/decrypt password.

so, for me key.txt is the same file loaded by Solr and by running the above command the pwd.enc file gets populated by:

U2FsdGVkX18xlQ6Mgtirfe7thJ3J17lUC9nWvDf9iU0=

Which is a base64 string, decoded it reveals a prefix Salted__ followed by the encrypted password binary blob. And it is the value for the password property in the dataSource tag. The command with enc -p returns more info:

salt=31950E8C82D8AB7D
key=6F489EE571236EC11CCD4ABB5A2DA3A3
iv =99907BAF93567BD934A03E9D720EE020

If I run the opposite command, to decrypt and verify the result, it works fine:

openssl enc -base64 -d -aes-128-cbc -in pwd.enc -pass file:./key.txt

returns hello as expected. However, when I attempt to run the DataImport handler in Solr I get this exception:

    Full Import failed:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: Error decoding password Processing Document # 1
    at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:271)
    at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:415)
    at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:474)
    at org.apache.solr.handler.dataimport.DataImporter.lambda$runAsync$0(DataImporter.java:457)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: Error decoding password Processing Document # 1
    at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:417)
    at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:330)
    at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:233)
    ... 4 more
Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: Error decoding password Processing Document # 1
    at org.apache.solr.handler.dataimport.JdbcDataSource.decryptPwd(JdbcDataSource.java:132)
    at org.apache.solr.handler.dataimport.JdbcDataSource.init(JdbcDataSource.java:75)
    at org.apache.solr.handler.dataimport.DataImporter.getDataSourceInstance(DataImporter.java:388)
    at org.apache.solr.handler.dataimport.ContextImpl.getDataSource(ContextImpl.java:100)
    at org.apache.solr.handler.dataimport.SqlEntityProcessor.init(SqlEntityProcessor.java:53)
    at org.apache.solr.handler.dataimport.EntityProcessorWrapper.init(EntityProcessorWrapper.java:77)
    at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:434)
    at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:415)
    ... 6 more
Caused by: java.lang.IllegalStateException: Bad password, algorithm, mode or padding; no salt, wrong number of iterations or corrupted ciphertext.
    at org.apache.solr.util.CryptoKeys.decodeAES(CryptoKeys.java:249)
    at org.apache.solr.util.CryptoKeys.decodeAES(CryptoKeys.java:195)
    at org.apache.solr.handler.dataimport.JdbcDataSource.decryptPwd(JdbcDataSource.java:130)
    ... 13 more
Caused by: javax.crypto.BadPaddingException: Given final block not properly padded
    at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:989)
    at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:845)
    at com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:446)
    at javax.crypto.Cipher.doFinal(Cipher.java:2165)
    at org.apache.solr.util.CryptoKeys.decodeAES(CryptoKeys.java:245)
    ... 15 more

In practice it says there is an error decoding the password. The decryptPwd() method is this:

  private Properties decryptPwd(Context context, Properties initProps) {
    String encryptionKey = initProps.getProperty("encryptKeyFile");
    if (initProps.getProperty("password") != null && encryptionKey != null) {
      // this means the password is encrypted and use the file to decode it
      try {
        try (Reader fr = new InputStreamReader(new FileInputStream(encryptionKey), UTF_8)) {
          char[] chars = new char[100];//max 100 char password
          int len = fr.read(chars);
          if (len < 6)
            throw new DataImportHandlerException(SEVERE, "There should be a password of length 6 atleast " + encryptionKey);
          Properties props = new Properties();
          props.putAll(initProps);
          String password = null;
          try {
            password = CryptoKeys.decodeAES(initProps.getProperty("password"), new String(chars, 0, len)).trim();
          } catch (SolrException se) {
            throw new DataImportHandlerException(SEVERE, "Error decoding password", se.getCause());
          }
          props.put("password", password);
          initProps = props;
        }
      } catch (IOException e) {
        throw new DataImportHandlerException(SEVERE, "Could not load encryptKeyFile  " + encryptionKey);
      }
    }
    return initProps;
  }

and is defined in:

solr-7.0.1/solr/contrib/dataimporthandler/src/java/org/apache/solr/handler/dataimport/JdbcDataSource.java

the BadPaddingException thrown by the decodeAES() method is defined inside:

solr-7.0.1/solr/core/src/java/org/apache/solr/util/CryptoKeys.java

And looks like this:

public static String decodeAES(String base64CipherTxt, String pwd) {
  int[] strengths = new int[]{256, 192, 128};
  Exception e = null;
  for (int strength : strengths) {
    try {
      return decodeAES(base64CipherTxt, pwd, strength);
    } catch (Exception exp) {
      e = exp;
    }
  }
  throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Error decoding ", e);
}

public static String decodeAES(String base64CipherTxt, String pwd, final int keySizeBits) {
  final Charset ASCII = Charset.forName("ASCII");
  final int INDEX_KEY = 0;
  final int INDEX_IV = 1;
  final int ITERATIONS = 1;
  final int SALT_OFFSET = 8;
  final int SALT_SIZE = 8;
  final int CIPHERTEXT_OFFSET = SALT_OFFSET + SALT_SIZE;

  try {
    byte[] headerSaltAndCipherText = Base64.base64ToByteArray(base64CipherTxt);

    // --- extract salt & encrypted ---
    // header is "Salted__", ASCII encoded, if salt is being used (the default)
    byte[] salt = Arrays.copyOfRange(
      headerSaltAndCipherText, SALT_OFFSET, SALT_OFFSET + SALT_SIZE);
    byte[] encrypted = Arrays.copyOfRange(
      headerSaltAndCipherText, CIPHERTEXT_OFFSET, headerSaltAndCipherText.length);

    // --- specify cipher and digest for evpBytesTokey method ---

    Cipher aesCBC = Cipher.getInstance("AES/CBC/PKCS5Padding");
    MessageDigest md5 = MessageDigest.getInstance("MD5");

    // --- create key and IV  ---

    // the IV is useless, OpenSSL might as well have use zero's
    final byte[][] keyAndIV = evpBytesTokey(
      keySizeBits / Byte.SIZE,
      aesCBC.getBlockSize(),
      md5,
      salt,
      pwd.getBytes(ASCII),
      ITERATIONS);

    SecretKeySpec key = new SecretKeySpec(keyAndIV[INDEX_KEY], "AES");
    IvParameterSpec iv = new IvParameterSpec(keyAndIV[INDEX_IV]);

    // --- initialize cipher instance and decrypt ---

    aesCBC.init(Cipher.DECRYPT_MODE, key, iv);
    byte[] decrypted = aesCBC.doFinal(encrypted);
    return new String(decrypted, ASCII);
  } catch (BadPaddingException e) {
    // AKA "something went wrong"
    throw new IllegalStateException(
      "Bad password, algorithm, mode or padding;" +
      " no salt, wrong number of iterations or corrupted ciphertext.", e);
  } catch (IllegalBlockSizeException e) {
    throw new IllegalStateException(
      "Bad algorithm, mode or corrupted (resized) ciphertext.", e);
  } catch (GeneralSecurityException e) {
    throw new IllegalStateException(e);
  }
}

I tested also by setting 6F489EE571236EC11CCD4ABB5A2DA3A3 which was the output of openssl for the key, but I get the same error. I restarted Solr at each change and also attempted to set a wrong path to see if it output a different error and it sorted Could not load encryptKeyFile... as expected. Is there something evidently wrong in what I'm doing? Do you have any suggestions?

Something that bothers me is the exception Caused by: javax.crypto.BadPaddingException: Given final block not properly padded message, which seems to lead to:

The thing about padding errors is that they are usually not errors in the padding but the wrong key or IV during decryption.

And the comment in the decodeAES() method:

// the IV is useless, OpenSSL might as well have use zero's

Above, instead, I see that openssl is defining IV with value 99907BAF93567BD934A03E9D720EE020 so I wonder if the problem is this. And if this can be caused by my system implementation of openssl (Ubuntu 16.04 right now).

Thank you for reading.

I'm going another direction here. It's improper to decode passwords today. The accepted method is to encrypt and store that result.

But how to check if the user's password is valid on login? Take the user's supplied input password, then encrypt and compare with the stored encrypted value.

In short, we NEVER want to see their password in the clear or stored in a decryptable database. Why? Read the news.

Hmm, no, here I'm not accepting passwords from users. It's a server side configuration step to connect Solr with MySQL. Solr is a search engine service developed by Apache. Here I'm setting the credentials into a configuration file, named data-config.xml to access the database. This file can store the database password in plain text or in an encrypted version.

I have an issue with the encrypted implementation. In practice it reads the value from the configuration file, but it does not decrypt and I wonder if it's caused by the IV value. I understand your point, but I cannot refactor this part, unless I'm missing something, also because resetting those values would mean to restart Solr, unless doing something like suggested (but not shown :D):

I'm not sure it's doable with the current version. Anyway, the methods I'm showing above are from the Solr code, I seeked them from the source version, to understand the error log.

As far as I know it can be added the Data Encryption (SSL) to the JDBC driver to enhance security, which will happen as soon this works, but in order to make the connection to the database, Solr has to decrypt the password stored in the configuration file.

Hi cereal,
Did you ever figure this out? I'm having the same problem:
"Error decoding password"
"Bad password, algorithm, mode or padding"

I've used this command to encrypt:
openssl enc -aes-128-cbc -a -salt -in [password]

...based on this documentation:
https://lucene.apache.org/solr/guide/7_0/uploading-structured-data-store-data-with-the-data-import-handler.html

I'm currently looking at comments in here:
https://issues.apache.org/jira/browse/SOLR-4392

Maybe this:
"If you create a key file with vi from scratch, you will get the EOL character, and that will break the encrypted password"?

OK, so @elyograg in the #solr IRC channel helped me figure this out.
When writing you decryption txt file use echo -n. -n ensures no EOL characters.

After trying the following unsuccessfully:
tr -d "\n\r" < decrypt.txt
... I deleted the file and re-echo'd with -n and that was my solution.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.