HashMap re-set question.

[new_2_java]

Hi all,

I have an HashMap with key/value pair. The program processes each element of the hashMap, and should mark the HashMap element as "processed". i.e. I have the follwoing key/value pair.

1/"090003453bc454"
2/"090003453bc455"
3/"090003453bc456"
and so on...

So, when the program process each element, it should mark the key to a negative one. So, each process element should look like the following.

-1/"090003453bc454"
-1/"090003453bc455"
-1/"090003453bc456"
and so on...

So, my question is, how can I set the key to (-1) after processing the elements in the HashMap?

Please advise...

Thanks,

[Ezzaral]

I can't see any reason that you would want to. It would render the map useless.

[new_2_java]

In second thaught, now I think, you are right. I would like to update the value and NOT the KEY

so the value of my HashMap would look like:

1/"Processed"
2/"Processed"
3/"Processed"
and so on...

Thanks

[Ezzaral]

It depends on what is being processed and what information you need to retain. Also, if your keys are just a sequenced index, you don't even need a Map implementation. A List would work fine. You can either mark the object processed or roll them off to another List of processed items.

[new_2_java]

Thanks for prompt reply,

I have ran a sample test and HashMap.put(..) does what I was looking for.

Basically, this is what my senario is:

I have a threaded application which spawns multiple threads, say 10. and each thread will operate on that HashMap list. So, soon as a thread chooses an element of the HashMap to operate on, it should mark the HashMap element as "Processed" so the other threads will not process the same element again. It's like simulating pasimestic locking mechanism.

Thanks again.

[Ezzaral]

Typically a Queue is used for that, often a BlockingQueue. Workers take items off the queue as they become available to process. Is there a reason that you need to leave them in a Map?

[new_2_java]

Actually, no. There's no particular reason. This was my initial thaughts. To elaborate alittle more, here I give alittle more specific details.

I am operating on a very huge amount of data (somewhere arround 200,000 elements). I can use sequentally, but I want to reduce the amount of time the program will take.

The skeleton of my program looks something like this: I think this will give a clearer picture of what I am trying to acheive.

public myClass extends Thread {
    public static void main(String[] arg) {
        myClass m = new myClass ();
        HashMap map = m.loadHashMap ();  // this will only load the HashMap wil key/value as mentioned above
        for (int i =0; i < 10; i++) {
            m.start();
        } 
    }
    
    public void run () {
        for (int j=0; j < map.size(); j++) {
            if (!map.get(new Integer(j)).toString().equalsIgnoreCase("Processed")) {
                map.put(new Integer(j), new String("Processed"));
                processElement(map.get(new Integer(j)).toString()); // this will the actual process
            }
        }
    }
}

So, what I am asking, basically, will this designe work? in other words, will each thread operate exactly on on element of the Hashmap, which is not marked yet as "Processed"?

I am open to ideas.

Thanks in advance...

[Ezzaral]

No, you have absolutely no synchronization there and will have all kinds of concurrency problems.

There are a lot of ways to structure what you are wanting to do, but you have to understand some basics about threading first. Additionally, if you are running on a single processor, starting 10 threads won't make it go faster - it will just have to time-slice the processing of each thread anyway.

[Ezzaral]

In case you do want to pursue it, start with the Java concurrency tutorial trail: http://java.sun.com/docs/books/tutorial/essential/concurrency/index.html

[new_2_java]

Ezzaral,

Thanks alot for the reference. I will deffinitly go over this tutorial.

Appreciate your help. I will do some reading and will update the thread accordingly,

Thanks

[Ezzaral]

I should actually qualify this part a little bit:

Additionally, if you are running on a single processor, starting 10 threads won't make it go faster - it will just have to time-slice the processing of each thread anyway.

If the process is CPU-bound, such as performing a lot of computations or loops, adding additional threads won't offer any improvement and may even make it slower due to the additional overhead of managing the threads themselves (synchronization, context switching, etc.). If the process performs some tasks that are constrained waiting on other resources like IO, multiple threads may offer improved performance even on single processor machines.

This article on thread pools and work queues covers some of these issues:
http://www.ibm.com/developerworks/library/j-jtp0730.html

[new_2_java]

Ezzaral,

The program is actually IO intensive. I am trying to write a migration program, which will read each content/metadata from a source repository and create its corresponding object (contetnt/metadata) to a target repository.

So, the program will be IO intensive as apposed to CPU, I believe.

So, will multi-thread help me in this senario?

Thanks for the suggestions.

[Ezzaral]

Perhaps, but again it all depends on the processes, the hardware, how you divide and synchronize the work, etc. There isn't a single global answer there.

[new_2_java]

I agree with what you said.

Can you take a look at the following and see if its better now?

public class MigrationUtility {
    private HashMap map = new HashMap ();
    
    public static void main(String[] args){
        MigrationUtility m = new MigrationUtility ();
        m.loadMap();
        
        for (int j=0; j < 10; j++) {
            new Thread(new MigrationUtilityThread(m.map)).start();
        }
    }
    
    private void loadMap () {
	// This will be populated properly.
	// For now its dummy values for test only.
        for (int i = 0;i < 50; i++) {
            map.put(new Integer(i), new String("Some Value" + i));
        }
    }
}

class MigrationUtilityThread extends Thread{
    private HashMap m = null;
    MigrationUtilityThread(HashMap map){
        m = map;
    }
    
    public void run(){
        for (int j=0; j < m.size(); j++) {
            if (!m.get(new Integer(j)).toString().equalsIgnoreCase("Processed")) {
                m.put(new Integer(j), new String("Processed"));
                processItem(this.getName());                
            }
        }
    }
    
    public synchronized void processItem(String name) {
	// process items here...
        System.out.println("Item processed with thread: "+ name );
    }
}

And this is the output of the program:

Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-0
Item processed with thread: Thread-16
Item processed with thread: Thread-4
Item processed with thread: Thread-6
Item processed with thread: Thread-8
Item processed with thread: Thread-2
Item processed with thread: Thread-12
Item processed with thread: Thread-18
Item processed with thread: Thread-10
Item processed with thread: Thread-14

I appreciate your comments.

Thanks,

[Ezzaral]

Read the section on work queues in the link that I posted above.

[new_2_java]

this is solved