Ashik’s IT Thoughts

March 13, 2010

Replacing $ in a String in Java

Filed under: Java — Tags: , , , , , , — ashikuzzaman @ 3:25 am

I faced an issue in my project where we read a password for an admin user first from a file which has a dummy or wrong password written in it. Then I read the password from database and replace the dummy password with the real password that I get from Database. So the code was something very simple like this (this is actually different from original).


String myXmlDoc = readALargeXmlAsString();
String realPassword = readFromDatabase();
String password = myXmlDoc.replaceAll("dummyPassword", realPassword);

All on a sudden we found that this replacement started failing after the password of that admin user was changed recently. Here is the exception log.

Exception in thread "main" java.lang.IllegalArgumentException: Illegal group reference
at java.util.regex.Matcher.appendReplacement(Matcher.java:706)
at java.util.regex.Matcher.replaceAll(Matcher.java:806)
at java.lang.String.replaceAll(String.java:2000)
at com.salesforce.test.StringReplaceTest.main(StringReplaceTest.java:34)

So I figured out from the log that it’s actually the replaceAll() pattern matching method that is causing the failure when you have $ in your replacement string (not in the pattern itself). So as per the first code example, if you had a $ character as one of the characters for realPassword, then it would throw the exception above. While the solution was to use a different admin for now who doesn’t have a password containing $ sign, I sat down to investigate it in detail and write a long term solution.

First I reproduced the issue in 3 different ways. Here is a code that will tell you depending on where you are putting the $ sign, the exception stack trace will be different. I give you the code and the compile and run instructions so that you can test it yourself.

package com.salesforce.test;

/**
* To compile: javac -d . StringReplaceTest.java
* To run: java com.salesforce.test.StringReplaceTest
* or, java com.salesforce.test.StringReplaceTest start
* or, java com.salesforce.test.StringReplaceTest middle
* or, java com.salesforce.test.StringReplaceTest end
* @authot ashik
*/
public class StringReplaceTest {

public static void main(String[] args) {

System.out.println("\nStringReplaceTest starts.....\n");

String firstStr = "I am a Java programmer working in USA. Chess is my hobby and here in USA lot of people play chess.";

System.out.println("firstStr before replacing = " + firstStr);

String positionPOfDollarSign = "";
if(args.length > 0) {
positionPOfDollarSign = args[0];
}
String secondStr = "";
if("start".equalsIgnoreCase(positionPOfDollarSign)) {
secondStr = firstStr.replaceAll("USA", "$PUT_A_VALUE123"); // illegal group reference
} else if("middle".equalsIgnoreCase(positionPOfDollarSign)) {
secondStr = firstStr.replaceAll("USA", "PUT_A_VALUE$123"); // String index out of range: 15
} else if("end".equalsIgnoreCase(positionPOfDollarSign)) {
secondStr = firstStr.replaceAll("USA", "PUT_A_VALUE123$"); // java.lang.IndexOutOfBoundsException: No group 1
} else {
secondStr = firstStr.replaceAll("USA", "PUT_A_VALUE123"); // no error
}

System.out.println("\nsecondStr after replacing firstStr = " + secondStr);

System.out.println("\nStringReplaceTest ends.....\n");
}

}

Developing the fix was a little tricky as replacing the $ and \ characters (these 2 are what makes trouble) using regular methods like Spring.split() or StringTokenizer class doesn’t work as those itself can’t process $ correctly. So I had to do my search and replace based on the String.indexOf() and String.substring(). Here is my fix and I would like to know your feedback on this. Please note that Apache StringUtils will be a very good resource to use here instead of trying to write the algorithm yourself.


package com.google.test;

import java.util.StringTokenizer;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

/**
* To compiple: javac -d . SfdcReplaceSubstring3.java
* To run: java com.google.test.SfdcReplaceSubstring3
*
* @author ashik
*/
public final class SfdcReplaceSubstring3 {

// private static String firstStr = "I am a Java programmer and a $Chess$ player working in USA. $Chess$ is my hobby and here in USA lot of people play $Chess$. USA had a great Chess player named Bobby Fischer.";

private static String firstStr = "Java, Apex, $Chess$ and Oracle - which one do you like? I guess Chess. If not $Chess$ then what else?";

private static String patternToSearch = "$Chess$";

private static String[] replacementStrFromDB = { null, "", " ", "PUT_A_VALUE123",
"PUT#A^VALUE!123?a+b/c>d", "PUT$A$VALUE123", "$PUT_A_VALUE123",
"PUT_A_$VALUE123", "PUT_A_VALUE123$", "PUT_A_VALUE$123",
"PUT_A_VALUE123$", "\\$PUT_A_VALUE123", "\\\\$PUT_A_VALUE123",
"\\PUT_A_VALUE123", "\\\\PUT_A_VALUE123", "\\PUT_A_VALUE$123",
"\\\\PUT_A_VALUE$123", "\\PUT_A_VALUE123$", "\\\\PUT_A_VALUE123$",
"$PUT_A_VALUE$123$" };

public static void main(String[] args) {
System.out.println("\nfirstStr before replacing = " + firstStr);
System.out.println("\npatternToSearch = " + patternToSearch);
// System.out.println("Direct replacement: " +
// matcher.replaceAll("PUT\\$A\\$VALUE123"));
for (int i = 0; i < replacementStrFromDB.length; i++)
try {
System.out.println("\nExecuted test#"
+ (i + 1)
+ ": "
+ "firstStr after replacing with "
+ replacementStrFromDB[i]
+ " = "
+ replace(firstStr, patternToSearch,
replacementStrFromDB[i]));
} catch (Exception e) {
System.out.println("\nExecuted test#" + (i + 1) + ": "
+ "Exception while replacing firstStr with "
+ replacementStrFromDB[i] + " = " + e.getMessage());
}
}

public static String replace(String text, String searchString,
String replacement) {
int start = 0;
int end = text.indexOf(searchString, start);
if (end == -1) {
return text;
}
int replLength = searchString.length();
StringBuilder buf = new StringBuilder();
while (end != -1) {
buf.append(text.substring(start, end)).append(replacement);
start = end + replLength;
end = text.indexOf(searchString, start);
}
buf.append(text.substring(start));
return buf.toString();
}

}

Update: This post attracted the attention of my friend Nitol wwith whom I have worked early in my career. He suggested to use Apache StringUtils for problems like this. Special thanks to him. It reminds me how many Apache open source libraries I have used earlier when I was in Bangladesh and after coming to USA, I don’t get time to sync up myself with the latest open source changes and often try to solve problems that has already been solved by numerous people (re-inventing the wheel). I will keep my eye on it from now on and allow myself to be a little lazy programmmer.

Advertisements

1 Comment »

  1. very informative..thanks a lot..james
    javajobs.net

    Comment by james smith — April 21, 2010 @ 1:19 am


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: